Outputting a saved hand-formed expression

ABSTRACT

Embodiments include an apparatus, device, system, computer-program product, and method. In an embodiment, a method is provided. The method includes receiving a request for a content of a computer-readable file. The method also includes searching the computer-readable file for the requested content, the computer-readable file including at least one representation of a hand-formed mark distributed to the computer-readable file in response to a machine-distinguishable tag associated with the hand-formed mark. The method further includes outputting the at least one representation of a hand-formed mark.

PRIORITY CLAIM, CROSS-REFERENCE TO RELATED APPLICATION, ANDINCORPORATION BY REFERENCE

The present application is related to, claims the earliest availableeffective filing date(s) from (e.g., claims earliest available prioritydates for other than provisional patent applications; claims benefitsunder 35 U.S.C. § 119(e) for provisional patent applications), andincorporates by reference in its entirety all subject matter of theherein listed application(s); the present application also claims theearliest available effective filing date(s) from, and also incorporatesby reference in its entirety all subject matter of any and all parent,grandparent, great-grandparent, etc. applications of the herein listedapplication(s). The United States Patent Office (USPTO) has published anotice to the effect that the USPTO's computer programs require thatpatent applicants reference both a serial number and indicate whether anapplication is a continuation or continuation in part. The presentapplicant entity has provided below a specific reference to theapplication(s) from which priority is being claimed as recited bystatute. Applicant entity understands that the statute is unambiguous inits specific reference language and does not require either a serialnumber or any characterization such as “continuation” or“continuation-in-part.” Notwithstanding the foregoing, applicant entityunderstands that the USPTO's computer programs have certain data entryrequirements, and hence applicant entity is designating the presentapplication as a continuation in part of its parent applications, butexpressly points out that such designations are not to be construed inany way as any type of commentary and/or admission as to whether or notthe present application contains any new matter in addition to thematter of its parent application(s).

1. For purposes of the USPTO extra-statutory requirements, the presentapplication constitutes a continuation in part of currently co-pendingUnited States patent application entitled ELECTRONIC ACQUISITION OF AHAND FORMED EXPRESSION AND A CONTEXT OF THE EXPRESSION, naming AlexanderJ. Cohen, Edward K. Y. Jung, Royce A. Levien, Robert W. Lord, Mark A.Malamud, and John D. Rinaldo, Jr., as inventors, filed Mar. 18, 2005,Ser. No. 11/083,567.

2. For purposes of the USPTO extra-statutory requirements, the presentapplication constitutes a continuation in part of currently co-pendingUnited States patent application entitled ACQUISITION OF A USEREXPRESSION AND A CONTEXT OF THE EXPRESSION, naming Alexander J. Cohen,Edward K. Y. Jung, Royce A. Levien, Robert W. Lord, Mark A. Malamud, andJohn D. Rinaldo, Jr., as inventors, filed Mar. 31, 2005, Ser. No.11/097,977.

3. For purposes of the USPTO extra-statutory requirements, the presentapplication constitutes a continuation in part of currently co-pendingUnited States patent application entitled ACQUISITION OF A USEREXPRESSION AND AN ENVIRONMENT OF THE EXPRESSION, naming Alexander J.Cohen, Edward K. Y. Jung, Royce A. Levien, Robert W. Lord, Mark A.Malamud, and John D. Rinaldo, Jr., as inventors, filed Mar. 31, 2005,Ser. No. 11/097,980.

4. For purposes of the USPTO extra-statutory requirements, the presentapplication constitutes a continuation in part of currently co-pendingUnited States patent application entitled SYSTEM AND METHOD FORELECTRONIC ASSOCIATION OF CONTENT AND CONTEXT, naming Alexander J.Cohen, Edward K. Y. Jung, Royce A. Levien, Robert W. Lord, Mark A.Malamud, and John D. Rinaldo, Jr., as inventors, filed Apr. 25, 2005,Ser. No. 11/114,245.

5. For purposes of the USPTO extra-statutory requirements, the presentapplication constitutes a continuation in part of currently co-pendingUnited States patent application entitled PERFORMING AN ACTION WITHRESPECT TO A HAND-FORMED EXPRESSION, naming Alexander J. Cohen, EdwardK. Y. Jung, Royce A. Levien, Robert W. Lord, Mark A. Malamud, and JohnD. Rinaldo, Jr., as inventors, filed May 25, 2005, Ser. No. ______\attorney docket number 0904-003-134A-000000\ filed contemporaneouslyherewith.

6. For purposes of the USPTO extra-statutory requirements, the presentapplication constitutes a continuation in part of currently co-pendingUnited States patent application entitled PERFORMING AN ACTION WITHRESPECT TO A HAND-FORMED EXPRESSION, naming Alexander J. Cohen, EdwardK. Y. Jung, Royce A. Levien, Robert W. Lord, Mark A. Malamud, and JohnD. Rinaldo, Jr., as inventors, filed May 25, 2005, Ser. No. ______\attorney docket number 0904-003-134B-000000\ filed contemporaneouslyherewith.

SUMMARY

An embodiment provides a method. The method includes receiving a requestfor a content of a computer-readable file. The method also includessearching the computer-readable file for the requested content, thecomputer-readable file including at least one representation of ahand-formed mark distributed to the computer-readable file in responseto a machine-distinguishable tag associated with the hand-formed mark.The machine-distinguishable tag being selected from at least twomachine-distinguishable tags. The method further includes outputting theat least one representation of a hand-formed mark. In addition to theforegoing, other method embodiments are described in the claims,drawings, and text forming a part of the present application.

Another embodiment provides a computer program product. The computerprogram product includes program instructions operable to perform aprocess in a computer system. The process includes receiving a requestfor information saved in a computer-readable file, the computer-readablefile including at least one representation of a hand-formed markdistributed to the computer-readable file in response to amachine-distinguishable tag associated with the hand-formed mark, themachine-distinguishable tag selected from at least twomachine-distinguishable tags. The process also includes searching thecomputer-readable file for the requested information, and outputting theat least one representation of a hand-formed mark. The computer programproduct also includes a computer-readable signal-bearing medium bearingthe program instructions. In addition to the foregoing, other computerprogram product embodiments are described in the claims, drawings, andtext forming a part of the present application.

A further embodiment provides an apparatus. The apparatus includes acomputing device having a computer storage medium and a list managerprogram. The apparatus includes a data recipient file saved on thecomputer storage medium and including at least one representation of ahand-formed mark received by the data recipient file in response to amachine-distinguishable tag associated with a hand-formed mark, themachine-distinguishable tag selected from at least twomachine-distinguishable tags. The list manager program includesinstructions operable to perform a process in the computing device. Theprocess includes receiving a request for a content of the data recipientfile, and searching the computer storage medium for the requestedcontent of the data recipient file. The process also includes outputtingthe at least one representation of a hand-formed mark. In addition tothe foregoing, other apparatus embodiments are described in the claims,drawings, and text forming a part of the present application.

An embodiment provides a device. The device includes a means forreceiving a request for a content of a computer-readable file. Thedevice includes a means for searching the computer-readable file for therequested content, the computer-readable file including at least onerepresentation of a hand-formed mark distributed to thecomputer-readable file in response to a machine-distinguishable tagassociated with the hand-formed mark, the machine-distinguishable tagbeing selected from at least two machine-distinguishable tags. Thedevice also includes a means for providing the at least onerepresentation of a hand-formed mark. In addition to the foregoing,other device embodiments are described in the claims, drawings, and textforming a part of the present application.

The foregoing is a summary and thus by necessity containssimplifications, generalizations and omissions of detail. Consequently,those skilled in the art will appreciate that the summary isillustrative only and is not intended to be in any way limiting. Otheraspects, inventive features, and advantages of the devices and/orprocesses described herein, as defined solely by the claims, will becomeapparent in the detailed description set forth herein.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an exemplary system in which embodiments may beimplemented, including a thin computing device;

FIG. 2 illustrates another exemplary system in which embodiments may beimplemented, including a general-purpose computing device;

FIG. 3 illustrates an environment that includes partial view of anexemplary handheld writing device, and an exemplary documentenvironment;

FIG. 4 illustrates a partial view of an environment that includes anexemplary electronic pen, and the exemplary document environment of FIG.3;

FIG. 5 illustrates a partial view of an environment that includes anexemplary apparatus, and the exemplary document environment of FIG. 3;

FIG. 6 illustrates a partial view of an environment that includes anexemplary apparatus, and the exemplary document environment of FIG. 3;

FIG. 7 illustrates an exemplary operational flow;

FIG. 8 illustrates another exemplary operational flow;

FIG. 9 illustrates an alternative embodiment of the exemplaryoperational flow of FIG. 8;

FIG. 10 illustrates another alternative embodiment of the exemplaryoperational flow of FIG. 8;

FIG. 11 illustrates a partial view of an exemplary computer-programproduct that includes a computer program for executing a computerprocess in a handheld computing device;

FIG. 12 illustrates an environment that includes a partial view of anexemplary annotation system and a partial view of an exemplary item;

FIG. 13 illustrates an environment that includes a partial view of anexemplary annotation system and a partial view of an exemplary item;

FIG. 14 illustrates a partial view an alternative embodiment of theenvironment of FIG. 13;

FIG. 15 illustrates a partial view of an alternative embodiment of theenvironment of FIG. 13 that includes an exemplary head mountable systemand an exemplary item illustrated as an exemplary bound book;

FIG. 16 illustrates an environment that includes a partial view of anexemplary annotation system and a partial view of an exemplary item;

FIG. 17 illustrates an environment that includes a partial view of anexemplary annotation system and a partial view of an exemplary item;

FIG. 18 illustrates an environment that includes a partial view of anexemplary annotation system and a partial view of an exemplary item;

FIG. 19 illustrates an exemplary operational flow;

FIG. 20 illustrates an alternative embodiment of the exemplaryoperational flow of FIG. 19;

FIG. 21 illustrates another alternative embodiment of the exemplaryoperational flow of FIG. 19;

FIG. 22 illustrates a partial view of an exemplary wearable apparatus;

FIG. 23 illustrates an exemplary operational flow;

FIG. 24 illustrates an alternative embodiment of the operational flow ofFIG. 23;

FIG. 25 illustrates another alternative embodiment of the operationalflow of FIG. 23;

FIG. 26 illustrates a partial view of an exemplary wearable apparatus;

FIG. 27 illustrates an environment in which embodiments may beimplemented;

FIG. 28 illustrates another environment in which embodiments may beimplemented;

FIG. 29 illustrates an exemplary operational flow;

FIG. 30 illustrates an alternative embodiment of the exemplaryoperational flow of FIG. 29;

FIG. 31 illustrates another alternative embodiment of the exemplaryoperational flow of FIG. 29;

FIG. 32 illustrates a further alternative embodiment of the exemplaryoperational flow of FIG. 29;

FIG. 33 illustrates an alternative embodiment of the exemplaryoperational flow of FIG. 29;

FIG. 34 illustrates another alternative embodiment of the exemplaryoperational flow of FIG. 29;

FIG. 35 illustrates a further alternative embodiment of the exemplaryoperational flow of FIG. 29;

FIG. 36 illustrates a further alternative embodiment of the exemplaryoperational flow of FIG. 29;

FIG. 37 illustrates an alternative embodiment of the exemplaryoperational flow of FIG. 29;

FIG. 38 illustrates another alternative embodiment of the exemplaryoperational flow of FIG. 29;

FIG. 39 illustrates a further alternative embodiment of the exemplaryoperational flow of FIG. 29, which includes additional operations;

FIG. 40 illustrates a partial view of an exemplary apparatus;

FIG. 41 illustrates an exemplary operational flow;

FIG. 42 illustrates an exemplary operational flow;

FIG. 43 illustrates an alternative embodiment of the operational flow ofFIG. 42;

FIG. 44 illustrates an alternative embodiment of the operational flow ofFIG. 42;

FIG. 45 illustrates an exemplary operational flow;

FIG. 46 illustrates an alternative embodiment of the exemplaryoperational flow;

FIG. 47 illustrates a partial view of an exemplary computer programproduct that includes a computer program and a computer-readablesignal-bearing medium bearing the computer program;

FIG. 48 illustrates an exemplary system in which embodiments may beimplemented;

FIG. 49 illustrates a partial view of an exemplary apparatus;

FIG. 50 illustrates an exemplary operational flow;

FIG. 51 illustrates a partial view of an environment in whichembodiments may be implemented;

FIG. 52 illustrates an exemplary operation flow;

FIG. 53 illustrates an alternative embodiment of the exemplary operationflow of FIG. 52;

FIG. 54 illustrates another alternative embodiment of the exemplaryoperation flow of FIG. 52;

FIG. 55 illustrates a further alternative embodiment of the exemplaryoperation flow of FIG. 52;

FIG. 56 illustrates a partial view of an exemplary computer programproduct; and

FIG. 57 illustrates a partial view of an exemplary device.

DETAILED DESCRIPTION

In the following detailed description of exemplary embodiments,reference is made to the accompanying drawings, which form a parthereof. In the several figures, like referenced numerals identify likeelements. The detailed description and the drawings illustrate exemplaryembodiments. Other embodiments may be utilized, and other changes may bemade, without departing from the spirit or scope of the subject matterpresented here. The following detailed description is therefore not tobe taken in a limiting sense, and the scope of the claimed subjectmatter is defined by the appended claims.

FIG. 1 and the following discussion are intended to provide a brief,general description of an environment in which embodiments may beimplemented. FIG. 1 illustrates an exemplary system that includes a thincomputing device 20 that interfaces with an electronic device (notshown) that includes one or more functional elements 51. For example,the electronic device may include any item having electrical and/orelectronic components playing a role in a functionality of the item,such as a limited resource computing device, a digital camera, a cellphone, a printer, a refrigerator, a car, and an airplane. The thincomputing device 20 includes a processing unit 21, a system memory 22,and a system bus 23 that couples various system components including thesystem memory 22 to the processing unit 21. The system bus 23 may be anyof several types of bus structures including a memory bus or memorycontroller, a peripheral bus, and a local bus using any of a variety ofbus architectures. The system memory includes read-only memory (ROM) 24and random access memory (RAM) 25. A basic input/output system (BIOS)26, containing the basic routines that help to transfer informationbetween sub-components within the thin computing device 20, such asduring start-up, is stored in the ROM 24. A number of program modulesmay be stored in the ROM 24 and/or RAM 25, including an operating system28, one or more application programs 29, other program modules 30, andprogram data 31.

A user may enter commands and information into the computing device 20through input devices, such as a number of switches and buttons,illustrated as hardware buttons 44, connected to the system via asuitable interface 45. Input devices may further include atouch-sensitive display screen 32 with suitable input detectioncircuitry 33. The output circuitry of the touch-sensitive display 32 isconnected to the system bus 23 via a video driver 37. Other inputdevices may include a microphone 34 connected through a suitable audiointerface 35, and a physical hardware keyboard (not shown). In additionto the display 32, the computing device 20 may include other peripheraloutput devices, such as at least one speaker 38.

Other external input or output devices 39, such as a joystick, game pad,satellite dish, scanner or the like may be connected to the processingunit 21 through a USB port 40 and USB port interface 41, to the systembus 23. Alternatively, the other external input and output devices 39may be connected by other interfaces, such as a parallel port, game portor other port. The computing device 20 may further include or be capableof connecting to a flash card memory (not shown) through an appropriateconnection port (not shown). The computing device 20 may further includeor be capable of connecting with a network through a network port 42 andnetwork interface 43, and through wireless port 46 and correspondingwireless interface 47 may be provided to facilitate communication withother peripheral devices, including other computers, printers, and so on(not shown). It will be appreciated that the various components andconnections shown are exemplary and other components and means ofestablishing communications links may be used.

The computing device 20 may be primarily designed to include a userinterface having a character, key-based, other user data input via thetouch sensitive display 32 using a stylus (not shown). Moreover, theuser interface is not limited to an actual touch-sensitive panelarranged for directly receiving input, but may alternatively or inaddition respond to another input device, such as the microphone 34. Forexample, spoken words may be received at the microphone 34 andrecognized. Alternatively, the computing device 20 may be designed toinclude a user interface having a physical keyboard (not shown).

The device functional elements (not shown) are typically applicationspecific and related to a function of the electronic device. The devicefunctional elements are driven by a device functional element(s)interface 50, which coupled with the system bus 23. A functional elementmay typically perform a single well-defined task with little or no userconfiguration or setup, such as a refrigerator keeping food cold, a cellphone connecting with an appropriate tower and transceiving voice ordata information, and a camera capturing and saving an image.

FIG. 2 illustrates another exemplary system in which embodiments may beimplemented. FIG. 2 illustrates an electronic device that may correspondin whole or part to a general-purpose computing device, shown as acomputing system environment 100. Components of the computing systemenvironment 100 may include, but are not limited to, a computing device110 having a processing unit 120, a system memory 130, and a system bus121 that couples various system components including the system memoryto the processing unit 120. The system bus 121 may be any of severaltypes of bus structures including a memory bus or memory controller, aperipheral bus, and a local bus using any of a variety of busarchitectures. By way of example, and not limitation, such architecturesinclude Industry Standard Architecture (ISA) bus, Micro ChannelArchitecture (MCA) bus, Enhanced ISA (EISA) bus, Video ElectronicsStandards Association (VESA) local bus, and Peripheral ComponentInterconnect (PCI) bus, also known as Mezzanine bus.

The computing system environment 100 typically includes a variety ofcomputer-readable media products. Computer-readable media may includeany media that can be accessed by the computing device 110 and includeboth volatile and nonvolatile media, removable and non-removable media.By way of example, and not of limitation, computer-readable media mayinclude computer storage media and communications media. Computerstorage media includes both volatile and nonvolatile, removable andnon-removable media implemented in any method or technology for storageof information such as computer-readable instructions, data structures,program modules, or other data. Computer storage media include, but arenot limited to, random-access memory (RAM), read-only memory (ROM),electrically erasable programmable read-only memory (EEPROM), flashmemory, or other memory technology, CD-ROM, digital versatile disks(DVD), or other optical disk storage, magnetic cassettes, magnetic tape,magnetic disk storage, or other magnetic storage devices, or any othermedium which can be used to store the desired information and which canbe accessed by the computing device 110. Communications media typicallyembody computer-readable instructions, data structures, program modules,or other data in a modulated data signal such as a carrier wave or othertransport mechanism and include any information delivery media. The term“modulated data signal” means a signal that has one or more of itscharacteristics set or changed in such a manner as to encode informationin the signal. By way of example, and not limitation, communicationsmedia include wired media such as a wired network and a direct-wiredconnection and wireless media such as acoustic, RF, optical, andinfrared media. Combinations of any of the above should also be includedwithin the scope of computer-readable media.

The system memory 130 includes computer storage media in the form ofvolatile and nonvolatile memory such as ROM 131 and RAM 132. A basicinput/output system (BIOS) 133, containing the basic routines that helpto transfer information between elements within the computing device110, such as during start-up, is typically stored in ROM 131. RAM 132typically contains data and program modules that are immediatelyaccessible to or presently being operated on by processing unit 120. Byway of example, and not limitation, FIG. 2 illustrates an operatingsystem 134, application programs 135, other program modules 136, andprogram data 137. Often, the operating system 134 offers services toapplications programs 135 by way of one or more application programminginterfaces (APIs) (not shown). Because the operating system 134incorporates these services, developers of applications programs 135need not redevelop code to use the services. Examples of APIs providedby operating systems such as Microsoft's “WINDOWS” are well known in theart.

The computing device 110 may also include other removable/non-removable,volatile/nonvolatile computer storage media products. By way of exampleonly, FIG. 2 illustrates a non-removable non-volatile memory interface(hard disk interface) 140 that reads from and writes to non-removable,non-volatile magnetic media, a magnetic disk drive 151 that reads fromand writes to a removable, non-volatile magnetic disk 152, and anoptical disk drive 155 that reads from and writes to a removable,non-volatile optical disk 156 such as a CD ROM. Otherremovable/nonremovable, volatile/non-volatile computer storage mediathat can be used in the exemplary operating environment include, but arenot limited to, magnetic tape cassettes, flash memory cards, DVDs,digital video tape, solid state RAM, and solid state ROM. The hard diskdrive 141 is typically connected to the system bus 121 through anon-removable memory interface, such as the interface 140, and magneticdisk drive 151 and optical disk drive 155 are typically connected to thesystem bus 121 by a removable non-volatile memory interface, such asinterface 150.

The drives and their associated computer storage media discussed aboveand illustrated in FIG. 2 provide storage of computer-readableinstructions, data structures, program modules, and other data for thecomputing device 110. In FIG. 2, for example, hard disk drive 141 isillustrated as storing an operating system 144, application programs145, other program modules 146, and program data 147. Note that thesecomponents can either be the same as or different from the operatingsystem 134, application programs 135, other program modules 136, andprogram data 137. The operating system 144, application programs 145,other program modules 146, and program data 147 are given differentnumbers here to illustrate that, at a minimum, they are differentcopies. A user may enter commands and information into the computingdevice 110 through input devices such as a microphone 163, keyboard 162,and pointing device 161, commonly referred to as a mouse, trackball, ortouch pad. Other input devices (not shown) may include a joystick, gamepad, satellite dish, and scanner. These and other input devices areoften connected to the processing unit 120 through a user inputinterface 160 that is coupled to the system bus, but may be connected byother interface and bus structures, such as a parallel port, game port,or a universal serial bus (USB). A monitor 191 or other type of displaydevice is also connected to the system bus 121 via an interface, such asa video interface 190. In addition to the monitor, computers may alsoinclude other peripheral output devices such as speakers 197 and printer196, which may be connected through an output peripheral interface 195.

The computing system environment 100 may operate in a networkedenvironment using logical connections to one or more remote computers,such as a remote computer 180. The remote computer 180 may be a personalcomputer, a server, a router, a network PC, a peer device, or othercommon network node, and typically includes many or all of the elementsdescribed above relative to the computing device 110, although only amemory storage device 181 has been illustrated in FIG. 2. The logicalconnections depicted in FIG. 2 include a local area network (LAN) 171and a wide area network (WAN) 173, but may also include other networkssuch as a personal area network (PAN) (not shown). Such networkingenvironments are commonplace in offices, enterprise-wide computernetworks, intranets, and the Internet.

When used in a LAN networking environment, the computing systemenvironment 100 is connected to the LAN 171 through a network interfaceor adapter 170. When used in a WAN networking environment, the computingdevice 110 typically includes a modem 172 or other means forestablishing communications over the WAN 173, such as the Internet. Themodem 172, which may be internal or external, may be connected to thesystem bus 121 via the user input interface 160, or via anotherappropriate mechanism. In a networked environment, program modulesdepicted relative to the computing device 110, or portions thereof, maybe stored in a remote memory storage device. By way of example, and notlimitation, FIG. 2 illustrates remote application programs 185 asresiding on memory device 181. It will be appreciated that the networkconnections shown are exemplary and other means of establishing acommunications link between the computers may be used.

In the description that follows, certain embodiments may be describedwith reference to acts and symbolic representations of operations thatare performed by one or more computing devices, such as computing device10 of FIG. 2. As such, it will be understood that such acts andoperations, which are at times referred to as being computer-executed,include the manipulation by the processing unit of the computer ofelectrical signals representing data in a structured form. Thismanipulation transforms the data or maintains them at locations in thememory system of the computer, which reconfigures or otherwise altersthe operation of the computer in a manner well understood by thoseskilled in the art. The data structures in which data is maintained arephysical locations of the memory that have particular properties definedby the format of the data. However, while an embodiment is beingdescribed in the foregoing context, it is not meant to be limiting asthose of skill in the art will appreciate that the acts and operationsdescribed hereinafter may also be implemented in hardware.

FIG. 2 illustrates an example of a suitable environment on whichembodiments may be implemented. The computing system environment 100 ofFIG. 2 is an example of a suitable environment and is not intended tosuggest any limitation as to the scope of use or functionality of anembodiment. Neither should the environment be interpreted as having anydependency or requirement relating to any one or combination ofcomponents illustrated in an exemplary operating environment.

Embodiments may be implemented with numerous other general-purpose orspecial-purpose computing devices and computing system environments orconfigurations. Examples of well-known computing systems, environments,and configurations that may be suitable for use with an embodimentinclude, but are not limited to, personal computers, server computers,handheld or laptop devices, personal digital assistants, multiprocessorsystems, microprocessor-based systems, set top boxes, programmableconsumer electronics, network, minicomputers, mainframe computers, anddistributed computing environments that include any of the above systemsor devices.

Embodiments may be described in a general context of computer-executableinstructions, such as program modules, being executed by a computer.Generally, program modules include routines, programs, objects,components, data structures, etc., that perform particular tasks orimplement particular abstract data types. An embodiment may also bepracticed in distributed computing environments where tasks areperformed by remote processing devices that are linked through acommunications network. In a distributed computing environment, programmodules may be located in both local and remote computer storage mediaincluding memory storage devices.

The following includes a series of illustrations depictingimplementations of processes. For ease of understanding, certainillustrations are organized such that the initial illustrations presentimplementations via an overall “big picture” viewpoint and thereafterthe following illustrations present alternate implementations and/orexpansions of the “big picture” illustrations as either sub-steps oradditional steps building on one or more earlier-presentedillustrations. This style of presentation utilized herein (e.g.,beginning with a presentation of a illustration(s) presenting an overallview and thereafter providing additions to and/or further details insubsequent illustrations) generally allows for a rapid and easyunderstanding of the various process implementations.

FIG. 3 illustrates an environment 200 that includes partial view ofexemplary handheld writing device 210, and an exemplary documentenvironment 280. In an embodiment, the exemplary document environment280 (hereafter “document 280”) may include a single sheet, a part of apage, a single page, a poster, a display, multiple pages, a chapter, avolume, and/or a file. In a further embodiment, the document may includea book, such as a bound volume, a box, such as a retail package, acontainer, a carton, a label, a currency bill or note, and/or check. Inanother embodiment, the document may include an article of clothing, anitem, a structure, a vehicle, and/or an indicator, such as a temperaturegauge or speedometer. In an embodiment, a document includes any thinghaving a human recognizable content.

The document 280 includes a content portion 282. The content portion 282may include a word 286, illustrated as a word “Rhett,” a phrase 284,illustrated as the phrase “Frankly my dear, I don't give a damn,” and/orother printed or native content, schematically illustrated as a textualparagraph 288. In another embodiment, the content portion may include atextual content portion. For example, a textual content portion mayinclude words printed on a page, such as a page of a book, and/or wordsprinted on a cover or a spine of a book. In a further embodiment, thecontent portion may include a human language content, for example, textprinted in roman alphabet characters, math symbols, and/or scriptcharacters such as Asian language characters. In another embodiment, thecontent portion may include a human readable content, and/or graphicalcontent. In an embodiment, the content portion may include a graphicalcontent having a meaning or a significance to a human, such as apicture, a drawing, a figure, a photograph, an illustration, a diagram,a doodle, a decoration, a sketch, and/or a portrait. In anotherembodiment, the content portion may be arbitrary and not recognizable byan individual human reader. In a further embodiment, the content portionmay include any content primarily useful to a human reader. For example,a content portion may include a textual and/or graphical label on aretail article, such as clothing, a retail package, or a carton. Acontent portion may include an address written on shipping label or anaddress written on a side of a building. A content portion may includean odometer reading on a vehicle or a temperature reading on athermometer. In another embodiment, the content portion may include anincidental element not primarily employed by a human reader but thataids in distinguishing the document. For example, an incidental elementmay include a coffee stain, a torn page, and/or a physical attribute ofthe document. In an embodiment, a content portion may include a humanand/or machine recognizable characteristic form or shape, such a pair ofskis, a car, and/or a dress.

In an embodiment, the content portion 282 of the document 280 may bedisplayed on a surface 281, such as a paper surface, a surface of asheet of paper, a surface of a newspaper, a surface of a book, and/or asurface of a poster. In a further embodiment, the surface may include asurface of a box, a surface of a container, a surface of a carton, asurface of a label, a surface of a currency bill or note, and/or asurface of a check. In another embodiment, a surface may include asurface of an article of clothing, a surface of an item, a surface of astructure, a surface of a vehicle, and/or a surface of an indicator. Inan embodiment, a surface may include content located in at least twoplanes. In another embodiment, a surface may include a planar surface.In an embodiment, the content portion 282 may be electronicallydisplayed, such as a text displayed through the surface 281, and/or atext projected on the surface. The surface electronically displaying thecontent portion may include a surface of a computer monitor, a surfaceof a television screen, and/or a surface of an e-paper,

The exemplary handheld writing device 210 includes a writing element220. In an embodiment, the writing element may include an active writingelement, such as an ink cartridge operable to discharge a marking fluidonto a surface of a document. The active writing element may include areplaceable ink cartridge. In another embodiment, the active writingelement may include a pencil lead. In a further embodiment, the writingelement may include a non-marking element, such as non-marking stylus.

The writing device 210 also includes a writing detector module 230operable to generate information indicative of a handwriting movement250 by the writing element 220. The handwriting movement is illustratedas “My favorite movie line.” The writing detector module 230 may beimplemented in any manner, including software, hardware, firmware,and/or a combination thereof. An embodiment of the writing detectormodule 230 may be implemented using reflected light, optical image,sonar, ultrasound, and/or accelerometer detection of a handwritingmovement. The handwriting movement may be formed in response to handmovements. In an alternative embodiment, the handwriting movement may beformed in response to movements of another part of a body, such as bymovement of a user's head while holding the writing device in theirmouth.

In an embodiment, “information” includes data that may be processed,stored, or transmitted by a computing device. In another embodiment,“information” includes a meaning of data as may be interpreted by ormeaningful to people. In an embodiment, “data” includes a piece orpieces of information. Data may include facts, quantities, characters,and/or symbols. Data may be subject to operations by a computer, saved,and transmitted by electrical signals.

In an embodiment, the writing detector module 230 may be implemented byilluminating a surface 281 of the document 280 from the handheld writingdevice 210, and determining the handwriting movement 250 of the writingelement 220 in response to reflected illumination from the surface, suchas more fully described in U.S. Pat. No. 6,741,335, titled OPTICALTRANSLATION MEASUREMENT, issued on May 25, 2004, Kinrot, et al. asinventor; U.S. Pat. No. 6,452,683, titled OPTICAL TRANSLATIONMEASUREMENT, issued on Sep. 17, 2002, Kinrot, et al. as inventor; andU.S. Pat. No. 6,330,057, titled OPTICAL TRANSLATION MEASUREMENT, issuedon Dec. 11, 2001, Lederer, et al. as inventor, each of which isincorporated herein by reference in their entirety for all that theydisclose and teach. In another embodiment, the writing detector modulemay be implemented by capturing images of a pattern in a writing baseand handwriting movements of the pen determined by calculating positionsof the writing element based on the images of the writing base, such asmore fully described in U.S. Pat. No. ______ (published as US2004/0179000) titled ELECTRONIC PEN, MOUNTING PART THEREFOR AND METHODOF MAKING THE PEN, filed Jun. 26, 2002, under application Ser. No.10/179,949, Fermgard, et al. as inventor, which is incorporated hereinby reference in its entirety for all that is discloses and teaches. In afurther embodiment, the writing detector module may be implemented usingultrasound to track a handwriting movement of a writing element, such asmore fully described in U.S. Pat. No. ______ (published as US2003/0173121) titled DIGITZER PEN filed Mar. 18, 2002, under applicationSer. No. 10/098,390, Zloter, et al. as inventor, which is incorporatedherein by reference in its entirety for all that it discloses andteaches. In an embodiment, the writing detector module may beimplemented using a sensor package that measures velocity, acceleration,and angular acceleration in a handheld writing device, such as morefully described in U.S. Pat. No. ______ (published as US 2004/0260507)titled 3D INPUT APPARATUS AND METHOD THEREOF, filed Jun. 17, 2004 underapplication Ser. No. 10/868,959, Chang, et al. as inventor, which isincorporated herein by reference in its entirety for all that itdiscloses and teaches. In another embodiment, the writing detectormodule may be implemented using an image processing approach thatcaptures and saves segments of a writing movement as a separate image.The final output is a combination of multiple images superimposed oneach other.

In another embodiment, the writing detector module 230 may generateinformation indicative of a handwriting movement 250 by capturing imagesof the handwriting movement, and stitching the images together.

In an embodiment, the writing detector module 230 may include anelectrical circuit 232 operable to generate information indicative of ahandwriting movement of the writing element. In a further embodiment,the writing detector module 230 may include operability to generateinformation indicative of a movement of the writing element generated byan activity of writing by hand. In another embodiment, the writingdetector module may include operability to generate informationindicative of a movement of the writing element in response to a handgenerated activity.

In a further embodiment, the writing detector module 230 may generateraw handwriting movement information. In another embodiment, the writingdetector module may process the raw handwriting movement information.

The writing device 210 also includes a context detector module 240operable to generate information indicative of a content portion of adocument proximate to the handwriting movement of the writing element.In an embodiment, a content portion of a document proximate to thehandwriting may include a content portion separated from the handwritingmovement of less than approximately one to two inches. In anotherembodiment, a content portion of a document proximate to the handwritingmovement may include a content portion and a handwriting movement bothon a page. In a further embodiment, a content portion of a documentproximate to the handwriting movement may include a content portion on abody that includes the handwriting movement. In an inanimate example,the content portion may be a book title and/or author printed on a bookspine or cover, and the handwriting movement being on a page of thebook. In an animate example, the content portion may include a portionof a human body, such as a skin surface having a landmark, and thehandwriting movement being on the skin surface and proximate to thelandmark, such a surgeon marking incision lines before beginning aprocedure.

The context detector module 240 may be implemented in any manner,including software, hardware, firmware, and/or any combination thereof.In an embodiment, the context detector module 240 may include anelectrical circuit 242 operable to generate information indicative of acontent portion 282 of a document proximate to the handwriting movementof the writing element 220, such as the phrase 284 and/or the word 286.An embodiment includes a context detector module operable to generateinformation indicative of a content portion of a displayed documentproximate to the handwriting movement of the writing element. A furtheralternative embodiment includes a context detector module operable togenerate information indicative of a content portion of anelectronically displayed document proximate to the handwriting movementof the writing element. Another embodiment includes a context detectormodule operable to generate information indicative of a printed contentportion of a document proximate to the handwriting movement of thewriting element. An embodiment includes a context detector moduleoperable to generate information indicative of a printed content portionof a paper-based document proximate to the handwriting movement of thewriting element. A further alternative embodiment includes a contextdetector module operable to generate information indicative of a contentportion of an e-paper document proximate to the handwriting movement ofthe writing element.

In an embodiment, the context detector module 240 may include an imagecapture device 246 having a lens 248. The image capture device 246includes operability to capture at least one image of the contentportion 282 proximate to the handwriting movement 250, such as thephrase 284 and/or the word 286. The image capture device 246 and thelens 248 may have any location with respect to the handheld writingdevice 210 suitable for capturing at least one image of a contentportion proximate to the handwriting movement, including within an outerperiphery of the handheld writing device, and/or outside the outerperiphery. In another embodiment, the image capture device 246 includesoperability to capture image information of the displayed contentportion proximate to the handwriting movement with a resolutionsufficient to render the image information into textural information byan optical character resolution process.

In a further embodiment, the image capture device 246 includesoperability to capture at least one image of a displayed content portionas a writer moves the writing element 220 of the handheld writing device210 toward the document 280. In another embodiment, the image capturedevice includes operability to capture at least one image of a displayedcontent portion as a writer imparts a handwriting movement to thewriting element. In a further embodiment, the image capture deviceincludes operability to capture at least one image of a displayedcontent portion as a writer moves the writing element of the handheldwriting device away from the document. The lens 248 may include a useradjustable orientation allowing a user to optimize image capture.

In an embodiment, the context detector module 240 may include a useractivatable switch (not shown) allowing a user to select when theinformation indicative of a content portion of a document element willbe captured. For example, the image capture device 246 may include auser activatable switch allowing a user to select when an image of acontent portion is obtained. A user may aim the lens 248 at a contentportion and capture an image by activating the switch. The switch mayinclude a hardware switch, a motion sensitive switch, and/or a voiceactivated switch.

In an embodiment, the context detector module 240 may further includeoperability to acquire contextual information from another source, andto use that contextual information to generate information indicative ofa content portion of a document proximate to the handwriting movement ofthe writing element. For example, the handwriting movement may occurproximate to a surface or a device operable to generate contextualinformation indicative of a content portion of a document proximate tothe handwriting movement. The surface may include a touch sensitivescreen coupled to a circuit operable to generate the contextualinformation. Alternatively, the handwriting movement may occur proximateto a surface monitored by an ultrasonic system coupled to a circuitoperable to generate contextual information indicative of a contentportion of a document proximate to the handwriting movement.

In a further embodiment, the context detector module 240 may generateraw information indicative of a content portion of a document proximateto the handwriting movement of the writing element. In anotherembodiment, the context detector module may process the raw contextportion information.

In another embodiment, the context detector module 240 may includeoperability to generate information indicative of a content portion of adocument proximate to the handwriting movement using other technologies.For example, information indicative of a content portion may begenerated using data from a radio frequency identification device (RFID)associated with the document, a global positioning satellite system,and/or other data signal.

In an alternative embodiment, the handheld writing device 210 mayinclude a recognition module 290. The recognition module includesoperability to generate information indicative of a human readablecontent in response to the information indicative of the hand drivenmovement of the writing element. For example, in an embodiment, therecognition module may determine a presence of human readable content inthe information indicative of the hand driven or handwriting movement250 generated by the writing detector module 230, recognize the humanreadable content as “My favorite movie line,” and generate informationindicative of that content. In another embodiment, the recognitionmodule may determine a presence of human readable content, recognize thehandwriting movements as forming the human readable content as a scriptcharacter, and generate information indicative of that script character.

In an operation of an embodiment, a reader may wish to annotate thedocument 280 with a comment or sketch. The reader holds the handheldwriting device 210 in their hand in a manner similar to a conventionalpen or pencil, and begins handwriting an annotation using the writingelement 220 proximate to a content portion 282. The handheld writingdevice 210 captures a handwriting movement, or hand driven movement, ofthe writing element imparted by the user's hand to the writing device,and through the writing detector module 230, generates informationindicative of the handwriting movement. The handheld writing devicecaptures a context of the handwriting movement through the contextdetector module 240 by capturing at least one image of the contentportion of the document proximate to the handwriting movement. Thecontext may be captured before the handwriting movement, during thehandwriting movement, and/or after the handwriting movement. The contextdetector module generates information indicative of the content portionof the document proximate to the handwriting movement.

FIG. 4 illustrates a partial view of an environment 201 that includes anexemplary electronic pen 211, and the exemplary document environment 280of FIG. 3. The electronic pen 211 includes a generally elongated body217 and the writing element 220 extending from the generally elongatedbody. In an embodiment, the writing element may be configured to make avisible mark. A visible mark may include any mark visible to a humaneye. For example, a visible mark may include an ink mark, and/or apencil mark. A visible mark may include a letter, a character, a symbol,a line, and/or a figure.

The electronic pen 211 also includes a writing detector module 234operable to generate information indicative of a hand driven movement ofthe writing element. In an embodiment, a hand driven movement includes ahand-generated movement. In an alternative embodiment, the writingdetector module includes operability to generate information indicativeof a hand driven movement of the writing element in contact with thesurface 281 of a document 280. In another alternative embodiment, thewriting detector module includes operability to generate informationindicative of a hand driven movement of the writing element in contactwith the surface of a paper document. In a further embodiment, thewriting detector module includes operability to generate informationindicative of a hand driven movement of the writing element in contactwith a surface displaying an electronic document. The writing detectormodule may be implemented in any manner, including software, hardware,firmware, and/or a combination thereof. An embodiment of the writingdetector module may be implemented using reflected light, optical image,sonar, ultrasound, and/or accelerometer detection of the handwritingmovement. In an embodiment, the writing detector module may beimplemented in a manner at least substantially similar to the writingdetector module 230 of FIG. 3.

The electronic pen further includes a context detector module 244operable to generate information indicative of a content portion of adocument proximate to the hand driven movement of the writing element.In an alternative embodiment, the context detector module may includeoperability to generate information indicative of a content portion of apage of multiage document proximate to the handwriting movement of thewriting element. In another alternative embodiment, the context detectormodule may include operability to generate information indicative of ahand-generated writing movement of the writing element. The contextdetector module 244 may be implemented in any manner, includingsoftware, hardware, firmware, and/or any combination thereof. In anembodiment, the context detector module 244 may be implemented in amanner at least substantially similar to the writing detector module 240of FIG. 3.

In an alternative embodiment, the electronic pen 211 may include atleast one additional module. Additional modules may include acommunications module 260 and a storage module 270. In an embodiment,the communications module 260 includes operability to transmit at leastone of the signal indicative of a hand driven movement of a writingelement and the signal indicative of at least a portion of a documentproximate to the hand driven movement of the writing element. The term“signal” means at least one current signal, voltage signal, or signal.In another embodiment, the communications module includes operability totransmit in real time at least one of the information indicative of ahand driven movement of a writing element and the information indicativeof at least a portion of a document proximate to the hand drivenmovement of the writing element. In a further embodiment, thecommunications module includes operability to transmit at least one ofprocessed information corresponding to the information indicative of ahandwriting movement of a writing element and processed informationcorresponding to the information indicative of at least a portion of adocument proximate to the writing movement of the writing element.

The communications module 260 may transmit a signal. In an optionalembodiment, the communications module both receives and transmitssignals (transceiver). For example and without limitation, “signal”includes a current signal, voltage signal, magnetic signal, or opticalsignal in a format capable of being stored, transferred, combined,compared, or otherwise manipulated. The communications module mayinclude wireless, wired, infrared, optical, and/or other communicationstechniques. In an embodiment, the communications module may includeoperability for communication with a computing device, such as the thincomputing device 20 of FIG. 1, and/or the computing device 110 of FIG.2. The communications module may include an antenna for wirelesscommunication, a connection for wired connection, and/or an optical portfor optical communication.

The digital storage module 270 may include any suitable digital storagemedium. For example, a digital storage medium may include a computerstorage medium. The digital storage module includes operability to saveat least one of the information indicative of a hand driven movement ofa writing element and the information indicative of at least a portionof a document proximate to the hand driven movement of the writingelement. Information may be saved in any form or format, such as a rawdata form, or a processed data form. In conjunction with thecommunications module 260, information may be saved as generated andthen be available for uploading at a later time. For example,information may be stored, batched, and subsequently transmitted. Inanother example, information may be stored, and subsequently transmittedafter the electronic pen 211 is docked.

In operation of an embodiment, the electronic pen 211 acquires ahandwritten annotation and a context of the handwritten annotation in amanner at least substantially similar to the handheld writing device 210of FIG. 3. In addition, the electronic pen 211 may transmit signalsindicative of the handwritten annotation and the context of theannotation using the communication module 260. One or both of thesignals may be transmitted in real time as a reader writes anannotation. In an alternative embodiment, one or both of the signals maybe processed by a component of the electronic pen prior to transmission.In another alternative embodiment, the electronic pen may save one orboth of the signals in the digital storage medium 270. The saved signalmay be transmitted at a later time.

FIG. 5 illustrates a partial view of an environment 202 that includes anexemplary apparatus 212, and the exemplary document environment 280 ofFIG. 3. The apparatus 212 includes operability to mark the document 280,such as the handwriting movement 250, and to generate an electronicindicator of a position 289 of the mark relative to an existingcharacter, such as the word 286 “Rhett” of the document. The indicator289 of a position of the mark relative to an existing word 286 may beexpressed in any manner, including an indication in Cartesiancoordinates, and/or spherical coordinates. In an embodiment, theexisting character of the document includes at least one existing letterdisplayed by the document. In another embodiment, the existing characterof the document includes at least one existing word displayed by thedocument. In a further embodiment, the document includes at least oneexisting image displayed by the document.

In an alternative embodiment, the apparatus 212 includes a penconfigured to mark relative to an existing character, such as the word286, of a document 280 based, at least in part, on a marking position289 relative to the existing character of the document, wherein the markand an indicator of the marking position are saved within the pen.

FIG. 6 illustrates a partial view of an environment 203 that includes anexemplary apparatus 213, and the exemplary document environment 280 ofFIG. 3. The apparatus 213 includes a stylus 221 configured to write on asurface, such as the surface 281 of the document 280. The stylus may beat least similar to the stylus 220 of FIG. 3. The apparatus alsoincludes a position sensor 236 operable to generate informationindicative of handwriting movement 250 of the stylus relative to thesurface. The apparatus includes an imaging element 245 operable togenerate information representing at least a portion of the word 286displayed from the surface proximate to the handwriting movements.

FIG. 7 illustrates an exemplary operational flow 300. After a startoperation, the operational flow 300 moves to a content operation 310. Atthe content operation 310, a first information is generated in ahandheld device indicative of a handwriting movement of a writingelement physically associated with the handheld device. At the operation320, a second information is generated in the handheld deviceinformation indicative of a content portion of a document proximate tothe handwriting movement of the writing element. The operational flow300 then proceeds to an end operation.

FIG. 8 illustrates an exemplary operational flow 340. After a startoperation, the operational flow 340 moves to a writing operation 345. Atthe writing operation 345, at least a portion of a document is marked inresponse to a hand driven movement of a handheld marking device. At acontent operation 350, a first data set indicative of the marking isgenerated in the handheld marking device. At a context operation 355, asecond data set indicative of a content portion of the documentproximate to the marking is generated in the handheld marking device.

FIG. 9 illustrates an alternative embodiment of the exemplaryoperational flow 340 of FIG. 8. FIG. 9 illustrates an embodiment wherethe exemplary operation flow 340 may include at least one additionaloperation 360. The additional operation 360 may include an operation 362and an operation 364. At the operation 362, the first data setindicative of the marking is saved in a digital storage mediumphysically associated with the handheld device. At the operation 364,the second data set indicative of a content portion of the documentproximate to the marking is saved in a digital storage medium physicallyassociated with the handheld device.

FIG. 10 illustrates an alternative embodiment of the exemplaryoperational flow 340 of FIG. 8. FIG. 10 illustrates an embodiment wherethe exemplary operation flow 340 may include at least one additionaloperation 370. The additional operation 370 may include an operation372, an operation 374, and an operation 378. At the operation 372, thefirst data set indicative of the marking and the second data setindicative of a content portion of the document proximate to marking aretransmitted from the handheld marking device. At the operation 374, thefirst data set indicative of the marking is transmitted from thehandheld marking device. The operation 374 may include at least oneadditional operation, such as an operation 376. At the operation 376,processed data corresponding to the first data set indicative of themarking is transmitted from the handheld marking device. At theoperation 378, the second data set indicative of a content portion ofthe document proximate to marking is transmitted from the handheldmarking device. The operation 378 may include at least one additionaloperation, such as an operation 379. At the operation 379, processeddata corresponding to the second data set indicative of a contentportion of the document proximate to marking is transmitted from thehandheld marking device.

FIG. 11 illustrates a partial view of an exemplary computer-programproduct 400 that includes a computer program 404 for executing acomputer process in a handheld computing device. An embodiment of theexemplary computer-program product 400 may be provided using acomputer-readable medium 402, and includes computer executableinstructions. The computer product 400 encodes the computer program 404for executing on the handheld computing device a computer process. Thecomputer process includes generating in the handheld device a first dataset indicative of a handwriting movement of a writing element physicallyassociated with the handheld device, and generating in the handhelddevice a second data set indicative of a content portion of a documentproximate to the handwriting movement of the writing element. In analternative embodiment, the computer process 404 may further include anadditional process, such as a process 406, a process 408, and a process410. At the process 406, the first data set indicative of a handwritingmovement and the second data set indicative of a content portion of thedocument proximate to the handwriting movement are saved in a digitalstorage medium physically associated with the handheld device. At theprocess 408, the first data set indicative of the handwriting movementis transmitted. At the process 410, the second data set indicative of acontent portion of the document proximate to the handwriting movement istransmitted. The computer-readable medium 402 may include a computerstorage medium, which may be carried by a computer-readable carrier (notshown). The computer-readable medium 402 may include a communicationsmedium (not shown). The computer-program product 400 may be implementedin hardware, software, and/or firmware.

FIG. 12 illustrates an environment 500 that includes a partial view ofan exemplary annotation system and a partial view of an exemplary item.The exemplary annotation system is illustrated as an exemplary handheldsystem 510. The exemplary item is illustrated as a ski 560 having at atop surface 561, and two recognizable aspects printed and/or painted onthe surface. Recognizable aspects are illustrated as a text content “iSlalom” 562 and a trademark content “Head Ski” 564. Another recognizableaspect may include a recognizable ski shape, illustrated as arecognizable outer periphery 566, a recognizable electronic productcode, illustrated as an electronically readable bar code 568, and/or arecognizable dimension 565, illustrated as a length of 165 cm.

In a further embodiment, a recognizable aspect may include a textualcontent portion. For example, a textual content portion may includewords printed or displayed on a surface, such as a page of a book,and/or words printed on a cover or a spine of a book. In a furtherembodiment, a recognizable aspect may include a human language content,for example, text printed in roman alphabet characters, math symbols,and/or script characters such as Asian language characters. In anotherembodiment, a recognizable aspect may include a human readable content,and/or graphical content. In an embodiment, a recognizable aspect mayinclude a graphical content having a meaning or a significance to ahuman, such as a picture, a drawing, a figure, a photograph, anillustration, a diagram, a doodle, a decoration, a sketch, and/or aportrait. In another embodiment, a recognizable aspect may be arbitraryand recognizable by an individual human reader, and not recognizable byanother individual human reader. In a further embodiment, a recognizableaspect may include any content primarily useful to a human reader. Forexample, a recognizable aspect may include a textual and/or graphicallabel on a retail article, such as clothing, a retail package, or acarton. A recognizable aspect may include an address written on ashipping label or an address written on a side of a building. Arecognizable aspect may include dynamically displayed matter, such as analignment of an indicator needle with a speed value on a vehiclespeedometer, or a temperature reading on a thermometer. In anotherembodiment, a recognizable aspect may include an incidental element notprimarily employed by a human reader but that aids in distinguishing theitem. In an embodiment, a recognizable aspect may include any aspectrecognizable by a human, a computing device, a recognition programrunning on a computing device, a computer implement method, and/or amachine. In another embodiment, a recognizable aspect may include acharacteristic form and/or shape, such a pair of skates, a car, and/or adress. In an embodiment, an aspect may be recognizable locally inresponse to information indicative of a recognizable aspect of an item.In another embodiment, an aspect may be recognizable in response toinformation provided by a remote third-party responsive at least in partto information indicative of a recognizable aspect of an item.

The exemplary handheld system 510 includes a writing element 520operable to form a user expression. The user expression is illustratedas a handwritten user expression “Check the Internet for best skiprices” 550. In an embodiment, the writing element may include an activewriting element, such as an ink cartridge operable to discharge amarking fluid onto a surface of a document. The active writing elementmay include a replaceable ink cartridge. The active writing element mayinclude a disappearing ink. The active writing element may include apencil lead. In a further embodiment, the writing element may include anon-marking element, such as non-marking stylus. In an alternativeembodiment, the system may include a microphone 522 operable to capturean audible user expression, such as a sound, word, and/or sentence.

The handheld system 510 includes an annotating device 530 operable togenerate information indicative of a user expression associated with arecognizable aspect of the item 560. In an embodiment where the userexpression includes the handwritten user expression 550 visually orspatially associated with the recognizable trademark content “Head Ski”564 and/or the text content “i Slalom” 562 on the top surface 561, theannotating device includes operability to generate informationindicative of the handwritten user expression 550 formed by the writingelement 520. Where the exemplary item includes a retail item in a store,such as the ski 560, the writing element may include a non-markingwriting element. Alternatively, when the item includes a retail item ina store, a user may adhere a “Post It” type notepaper proximate to arecognizable aspect, handwrite on the “Post It” so as not to damage ormar the item, capture the handwritten expression, and then remove the“Post It.” In an embodiment where the user expression includes anaudible user expression, such as a spoken sentence “Check the Internetfor best ski prices,” the annotating device may include operability togenerate information indicative of the spoken sentence received by themicrophone 522.

The annotating device 530 may be implemented in any manner, includingsoftware, hardware, firmware, and/or a combination thereof. Theannotating device may be implemented in a manner at least substantiallysimilar to the writing detector module 230 of FIG. 3. In an embodiment,the annotating device may include operability to generate informationindicative of a user verbal expression (not shown) associated with therecognizable aspect of the item. The annotating device may includeoperability to generate information indicative of the handwritten userexpression “Check the Internet for best ski prices” 550 associated withthe recognizable aspect of the item. The annotating device may includeoperability to generate information indicative of a user hand drawnexpression associated with the recognizable aspect of the item. Theannotating device may include operability to generate informationindicative of a user gesture associated with the recognizable aspect ofthe item. The annotating device may include operability to generateinformation indicative of a user gesture formed at least in part by auser limb. The annotating device may include operability to generateinformation indicative of a user gesture formed with a user held stylus.

Continuing with FIG. 12, the handheld system 510 includes an annotationenvironment capture device 540 operable to generate informationindicative of a recognizable aspect of an item, such as the text content“i Slalom” 562. The annotation environment capture device may beimplemented in any manner, including software, hardware, firmware,and/or any combination thereof. The context-detecting device may beimplemented in a manner at least substantially similar to the contextdetector module 240 of FIG. 3. In an embodiment, the context-detectingdevice 540 may include operability to generate information indicative ofa recognizable aspect of an item in response to data acquired from atleast one of several technologies and/or data sources. For example, dataindicative of a recognizable aspect of an item may be acquired from datacorresponding to the recognizable aspect of the item, such as forexample, time of day, time of user expression, time period of userexpression, user-entered time, time bracket, date, location, presence ofanother person, presence of other items, temperature, elevation,bearing, and/or global position coordinates. By way of further example,data corresponding to the recognizable aspect of an item may be acquiredat least in part from a radio frequency identification device (RFID).The RFID may be directly associated with the aspect of item, associatedwith the item, and/or indirectly associated with the item.

In another embodiment, the context-detecting device 540 may include animage capture device 546 having a lens 548. The image capture deviceand/or the lens may be implemented in a manner at least substantiallysimilar to the image capture device 246 and/or lens 248 of FIG. 3. In anembodiment, the context-detecting device 540 may include auser-activatable switch 542 allowing a user to select when theinformation indicative of a recognizable aspect of an item will becaptured. For example, a user may aim the lens 548 at a recognizableaspect of an item and capture an image by activating the switch 542. Theswitch may include a hardware switch, a motion sensitive switch, and/ora voice activated switch. In a further embodiment, the context-detectingdevice may generate raw information indicative of a recognizable aspectof an item. In another embodiment, the context-detecting device mayprocess the raw information indicative of a recognizable aspect of anitem.

In an embodiment, the context-detecting device 540 may includeoperability to generate information indicative of a machine recognizableaspect of an item, such as the ski 560. A machine recognizable aspect ofthe ski may include a ratio of its length dimension 565 to its width(not shown). The context-detecting device may include operability togenerate information indicative of an optically recognizable aspect ofan item. The context-detecting device may include operability togenerate information indicative of a recognizable human readable contentof an item, such as the trademark content “Head Ski” 564. Thecontext-detecting device may include operability to generate informationindicative of a recognizable native text of an item, such as the textcontent “i Slalom” 562. The context-detecting device may includeoperability to generate information indicative of a recognizable shapeaspect of an item, such as the ski shape 566. The context-detectingdevice may include operability to generate information indicative of arecognizable dimensional aspect of an item, such as the length dimension565 of the ski 560.

In an alternative embodiment of the handheld system 510, an impactand/or pressure sensitive switch 521 may include a non-marking elementversion of the writing element 520. Tapping the non-marking element 520against a surface of an item, such as the surface 561, activates theswitch 521. In an embodiment, activation of the switch causes theannotating device 530 to record a verbal user expression. In anotherembodiment, activation of the switch causes the context-detecting device540 to acquire an image of a recognizable aspect of the item, such asthe trademark content “Head Ski” 564.

In a further embodiment, the context-detecting device 540 may include amodule operable to recognize data indicative of the recognizable aspectof an item. The module operable to recognize data indicative of therecognizable aspect of an item may include an artificial intelligencemodule operable to recognize data indicative of the recognizable aspectof an item. The module operable to recognize data indicative of therecognizable aspect of an item may include a pattern recognition moduleoperable to recognize data indicative of the recognizable aspect of anitem.

In an embodiment, the system may include a communications device (notshown). The communications device may include operability to communicateat least one of the information indicative of a recognizable aspect ofan item and the information indicative of a user expression associatedwith the recognizable aspect of the item. In another embodiment, thesystem may include a storage device (not shown). The storage device mayinclude operability to save at least one of the information indicativeof a recognizable aspect of an item and the information indicative of auser expression associated with the recognizable aspect of the item.

As used in this document, an “item” may include a thing, an article, anobject, an occurrence, a garment, a vehicle, a body, a person, a worldlymatter, and/or a state of an item. An item may include a representation.An item may be very small, such as would require aided vision for humanviewing, or an item may be very large, such as a building, an aircraft,vehicle, and/or a vessel that a human could walk through. An item may beanimate and/or inanimate.

A user expression may be associated by the user with an item or anaspect of an item in any manner. In an embodiment, the user expressionmay be physically associated with an aspect of an item by the user. Forexample, a user may associate a user expression and an aspect of an itemby writing the user expression on the item and proximate to the aspect.Writing proximate to the text content “i Slalom” 562 is an example of auser expression associated by the user with an aspect of an item. Inanother embodiment, the user expression may be temporally associatedwith an item or aspect of an item. For example, a user may temporallyassociate a user expression and an item or aspect of an item by speakingat a time when the item or aspect of the item is spatially proximate tothe user. In another embodiment, a user may associate a user expressionand an item or aspect of an item by a gesture. For example, a user maygesturally associate a user expression and an item or aspect of an itemby tapping on or pointing to the item or aspect of the item whilespeaking.

In operation of an embodiment, a user may wish to associate a userexpression with a recognizable aspect of an item. For example, a usersigning a printed credit card slip in a restaurant may wish to associateinformation indicative of their entries and the printed credit cardslip. In addition, a user may wish to associate information indicativeof the meal and the printed credit card slip. In this example, thehandheld system 510 generally having a shape of a common writing pen maybe used. A written user expression associated with the credit card slipmay include a user written entry for a tip, a user written entry for atotal charge, and a user written signature entry, made with the activewriting element 520 having an ink tip.

In an embodiment, the written entries are captured by the annotatingdevice 530. In another embodiment, images of the written entries arecaptured by the image capture device 546. Further, before, during,and/or after the writing, the user may verbally express a descriptionthe meal and the quality of the meal in association with the printedcredit card slip, and perhaps a description of a wine consumed duringthe meal. A user verbal expression is captured by the microphone 522.

Also, before, during, and/or after the writing, the image capture device546 captures an image indicative of at least one recognizable aspect ofthe printed credit card slip. A recognizable aspect of the credit cardslip may include a name of the restaurant, a date, a time, a charge formeals, a sever name, and/or a charge for wine.

The annotating device 530 generates information indicative of the user'sverbal expression(s) and/or the user's handwritten expression(s). Thecontext-detecting device 540 generates information indicative of arecognizable aspect of the credit card slip. The information indicativeof a recognizable aspect of the credit card slip and/or the informationindicative of the user expression may be saved in the handheld systemfor later communication to a computing device, such as a computer devicerunning an expense account program.

FIG. 13 illustrates an environment 600 that includes a partial view ofan exemplary annotation system and a partial view of an exemplary item.The exemplary annotation system is illustrated as an exemplary humanwearable annotation system 610, which is further illustrated as anexemplary head mountable system 612. The exemplary item is illustratedas an exemplary bound paper book 660. The exemplary book illustratesseveral exemplary recognizable aspects, including a cover 682, an authorname 686, illustrated as J. R. R. Tolkien, a cover title 684,illustrated as “The Silmarillion,” and an electronically readableproduct code 688, illustrated as a bar code. Further illustratedexemplary recognizable aspects include a page having a page number 666,illustrated as page number “59,” a page surface 661, and a binding 680having a binding title 683, illustrated as “Silmarillion.” The exemplarybook also illustrates an exemplary recognizable aspect that includes arecognizable text content 664 printed on and native to the page surface661 of the page number 59.

The exemplary head mountable system 612 includes a frame 614, whichincludes an earpiece 616 and an eye cover 618. In an optionalembodiment, the eye cover may be omitted. The frame includes aconfiguration to be worn in a manner similar to a pair of eyeglasses. Inan embodiment, the eye cover 618 includes a configuration that allows asubstantially unobstructed line of sight, illustrated as a eye gaze 692,between an eye 690 and an aspect of an item, illustrated as therecognizable text 664. In an embodiment, the frame provides a structurethat incorporates an annotating device 630, an annotation environmentcapture device 640, and a microphone 622 and associated sound capturecircuitry. In an optional embodiment, the frame may also provide astructure that incorporates a user-activatable switch 644. In anotherembodiment, the system 612 may include any configuration allowing it tobe worn on a head.

The annotation environment capture device 640 includes operability togenerate information indicative of a recognizable aspect of an item. Inan embodiment, annotation environment capture device includes a trackingmodule 642 operable to track the eye gaze 692 directed proximate to arecognizable aspect of the item. In another embodiment, the annotationenvironment capture device includes an image capture module 646 operableto acquire an image of a recognizable aspect of the item. In a furtherembodiment, the annotation environment capture device includes an imagecapture module 646 and a lens 648 operable to acquire an image of arecognizable aspect of the item in response to a signal indicative of areceived user command. The user command may include a verbal command, agestural command, and/or activation of a switch, such as the switch 644.The capture module 646 and the lens 648 direct a lens track 652 on arecognizable aspect of an item.

The annotation environment capture device 640 operable to generateinformation indicative of a recognizable aspect of an item may include amodule operable to recognize data indicative of the recognizable aspectof an item. The module operable to recognize data indicative of therecognizable aspect of an item may include an artificial intelligencemodule operable to recognize data indicative of the recognizable aspectof an item. The module operable to recognize data indicative of therecognizable aspect of an item may include a pattern recognition moduleoperable to recognize data indicative of the recognizable aspect of anitem.

The annotation environment capture device operable to generateinformation indicative of a recognizable aspect of an item may include atracking module operable to track an eye gaze directed proximate to therecognizable aspect of the item. The annotation environment capturedevice operable to generate information indicative of a recognizableaspect of an item may include a receiver circuit operable to receive asignal containing data indicative of a recognizable aspect of the item.The annotation environment capture device operable to generateinformation indicative of a recognizable aspect of an item may includean image capture module operable to acquire an image of a recognizableaspect of the item. The annotation environment capture device operableto generate information indicative of a recognizable aspect of an itemmay include an image capture module operable to acquire an image of arecognizable aspect of the item in response to a signal indicative of areceived user command.

The annotation environment capture device 640 may include operability togenerate information indicative of a machine recognizable aspect of anitem. In an embodiment, a machine recognizable aspect of an item mayinclude the author name 686, the cover title 684, the page number 666,the binding title 683, the electronically readable product code 688,and/or the text content 664. The annotation environment capture devicemay include operability to generate information indicative of anoptically recognizable aspect of an item. In an embodiment, an opticallyrecognizable aspect of an item may include the author name, the covertitle, the page number, the binding title, and/or the text content. Theannotation environment capture device may include operability togenerate information indicative of a human recognizable aspect of anitem. In an embodiment, a human recognizable aspect of an item mayinclude the author name, the cover title, the page number, the bindingtitle, and/or the text content. The annotation environment capturedevice may include operability to generate information indicative of arecognizable native text of an item. In an embodiment, a recognizablenative text of an item may include the author name, the cover title, thepage number, the binding title, and/or the text content. The annotationenvironment capture device may include operability to generateinformation indicative of a recognizable visual aspect of an item. In anembodiment, a recognizable aspect of an item may include an opticallyrecognizable author name, cover title, page number, binding title,and/or text content. The annotation environment capture device mayinclude operability to generate information indicative of a recognizableimage aspect of an item. In an embodiment, a recognizable image aspectmay include a picture, a figure, a drawing, and/or a graphic element(not shown) of the item. The annotation environment capture device mayinclude operability to generate information indicative of a recognizableaspect of an item. In an embodiment, a recognizable aspect may includeany recognizable aspect of the book. The annotation environment capturedevice may include operability to generate information indicative of arecognizable human readable content. In an embodiment, a recognizablehuman readable content may include the author name, the cover title, thepage number, the binding title, and/or the text content. Thecontext-detecting device operable to generate information indicative ofa recognizable aspect of an item may include a context-detecting deviceoperable to generate information indicative of a recognizable audioaspect of an item. For example, the context-detecting device may includean operability to generate information indicative of a recognizableexcerpt of a music piece, such as a song, or an excerpt of a speech.

The annotation environment capture device operable to generateinformation indicative of a recognizable aspect of an item may includean annotation environment capture device operable to generateinformation indicative of a recognizable aspect of an item in responseto an input received from the user. For example, an input received froma user may include an interaction with a user to select an item to whichthe user expression is associated from among at least two candidateitems in an image. By way of further example, an input received from auser may include an interaction with a user to select an aspect of anitem to which the user expression is associated from among at least twocandidate aspects of an item. The annotation environment capture deviceoperable to generate information indicative of a recognizable aspect ofan item may include an annotation environment capture device operable togenerate information indicative of a recognizable aspect of an item inresponse to an input responsive to a gesture by the user. For example,an input responsive to a user gesture may include a user pointing to anitem to which the user expression is associated from among at least twocandidate items in an environment.

The annotation environment capture device 640 operable to generateinformation indicative of a recognizable aspect of an item may includean annotation environment capture device operable to generateinformation indicative of a recognizable shape aspect of an item. Theannotation environment capture device operable to generate informationindicative of a recognizable aspect of an item may include an annotationenvironment capture device operable to generate information indicativeof a recognizable dimensional aspect of an item.

In a further embodiment, the annotation environment capture device 640may include a module operable to recognize data indicative of therecognizable aspect of an item. The module operable to recognize dataindicative of the recognizable aspect of an item may include anartificial intelligence module operable to recognize data indicative ofthe recognizable aspect of an item. The module operable to recognizedata indicative of the recognizable aspect of an item may include apattern recognition module operable to recognize data indicative of therecognizable aspect of an item.

The annotation environment capture device 640 may be implemented in anymanner, including software, hardware, firmware, and/or any combinationthereof. The annotation environment capture device may be implemented ina manner at least substantially similar to the context-detector module240 of FIG. 3.

Continuing with FIG. 13, the annotating device 630 includes operabilityto generate information indicative of a user expression associated withthe recognizable aspect of the item. In an embodiment, the annotatingdevice may include operability to generate information indicative of auser verbal expression associated with the recognizable aspect of theitem. In an embodiment, a user verbal expression may include a verbalsound, a word, at least two words, and/or a sentence. In anotherembodiment, a user verbal expression may include any vocal expression auser wishes to associate with the recognizable aspect of the item. In anembodiment, the annotating device includes operability to generateinformation indicative of a preformed user expression associated withthe recognizable aspect of the item. In a further embodiment, apreformed user verbal expression may include at least one wordrepresentative of at least two words. For example, a verbalized word“high” may represent a verbal expression of “this aspect is of highinterest to me.”

The annotating device 630 may be implemented in any manner, includingsoftware, hardware, firmware, and/or any combination thereof. Theannotating device may be implemented in a manner at least substantiallysimilar to the context-detector module 230 of FIG. 3.

In operation of an embodiment, a user may don the exemplary headmountable system 612 in a manner similar to eyeglasses. The person maydirect their eye gaze 692 through the eye cover 618 and at one or morerecognizable aspects of an item, such as for example, the text content664. Alternatively, the user's gaze 692 may be directed toward any otherrecognizable aspect of the book described above, for example, the title684. The tracking module 642 tracks the gaze 692 of the user's eye 690and aligns the lens track 652 of the lens 648 of the image capturemodule 646 with the recognizable text content. An image of the textcontent 664 may be captured. The capture may be automatic, such as inresponse to a predetermined time that the gaze 692 is directed towardthe recognizable text context, and/or such as in response to the usermaking a verbal expression associated with the recognizable textcontext. Alternatively, the capture may be in response the useractivating the switch 644 through a touch or verbal command. Thecontext-detector 640 generates information indicative of therecognizable text context responsive to the captured image.

Continuing with the exemplary operation, the user may utter a verbalexpression for connection or association with the recognizable aspect ofthe book 660, which in this example includes the recognizable textaspect 664. The verbal expression may be any verbal expression. Forexample, a verbal expression may include “This is really interestingstuff,” or “This statement contradicts a statement at page 12.” Theverbal expression is captured by the microphone 622. The annotatingdevice generates information indicative of the user expression inresponse to the captured verbal expression.

In an alternative embodiment, the exemplary human wearable system 610may comprise two separate human wearable elements. For example, theannotation environment capture device 640 may be carried in a firstelement wearable on a person's head and the annotation device 630carried in a second element wearable around the person's neck. In afurther embodiment, the annotating device may include an annotatingdevice having a configuration selected from a group consisting of ahandheld device, a wearable device, and a head mountable device. Theannotation environment capture device includes an annotation environmentcapture device having a configuration other than the selectedconfiguration of the annotating device. In another embodiment, theannotation environment capture device may include an annotationenvironment capture device having a configuration selected from a groupconsisting of a handheld device, a wearable device, and a head mountabledevice. The annotating device includes an annotating device having aconfiguration other than the selected configuration of the annotationenvironment capture device.

FIG. 14 illustrates a partial view an alternative embodiment of theenvironment 600 of FIG. 13. The exemplary head mountable system 612 mayinclude at least one additional device. An additional device may includea communications device 654 coupled with an antenna (not shown), and astorage device 656. The communications device includes operability tocommunicate at least one of the information indicative of a recognizableaspect of an item and the information indicative of a user expressionassociated with the recognizable aspect of the item. The storage deviceincludes operability to save at least one of the information indicativeof a recognizable aspect of an item and the information indicative of auser expression associated with the recognizable aspect of the item.

FIG. 15 illustrates a partial view of an alternative embodiment of theenvironment 600 of FIG. 13 that includes an exemplary head mountablesystem 613 and an exemplary item illustrated as an exemplary bound book661. The alternative embodiment of the environment 600 includes anexemplary distributed annotation system. The system 613 includes thecontext-detection module 640, a communications device 654 coupled withan antenna 655. The system 613 is operable to generate informationindicative of a recognizable aspect of the item, illustrated as the book661.

The book 661 includes microphone 689 and associated sound capturecircuitry, an annotating device 632, and an antenna 672 coupled with acommunications device (not shown). The annotating device 632 includesoperability to generate information indicative of a user expressionassociated with a recognizable aspect of the item, illustrated as thebook 661. The annotating device 632 is at least substantially similar tothe annotating device 630 of FIG. 13. The microphone 689 and associatedsound capture circuitry is at least substantially similar to themicrophone 622 and associated sound capture circuitry of FIG. 13. Atleast one of the communications devices may be configured as a receivercircuit, a transmission circuit, and/or a transceiver circuit.

In operation of an embodiment, a user may don and use the exemplary headmountable system 613 to generate information indicative of arecognizable aspect of an item in a manner at least substantiallysimilar to the system 612 of FIG. 13. Also, as described in conjunctionwith FIG. 13, the user utters a verbal expression for connection orassociation with the recognizable aspect of the book 660, which in thisexample includes the recognizable text aspect 664. In the embodimentillustrated in FIG. 15, the verbal expression is captured by themicrophone 689. The annotating device 632 generates informationindicative of the user expression in response to the captured verbalexpression. In one embodiment, one of the book 661 and the headmountable system 613 may transmit the respective information generatedby their device to the other. In another embodiment, the book and thesystem may transmit the respective information generated by theirdevices to a third device.

FIG. 16 illustrates an environment 700 that includes a partial view ofan exemplary annotation system and a partial view of an exemplary item.The exemplary annotation system is illustrated as an exemplary humanwearable system 610, which is further illustrated as an exemplary headmountable system 612. The exemplary item is illustrated as an exemplarybound paper book 660.

The system 612 includes a tracking module 743 carried in the frame 614and operable to track a user finger 710 and/or a user fingertip 712proximate to a recognizable aspect of the item, such as the recognizabletext content 664 printed on and/or native to the page surface 661. In anembodiment, the tracking module may include operability to track agesture formed by the user finger and/or the user fingertip.

In an alternative embodiment, the tracking module 743 includesoperability to track a stylus. For example, a tracked stylus may includea hand holdable stylus (not shown) proximate to a recognizable aspect ofthe item.

In operation, a user may place their fingertip 712 and/or their finger710 on the surface 661 of the page and proximate to the recognizabletext content 664. The tracking module 743 tracks the finger and/orfingertip, and upon occurrence of a predetermined condition acquiresdata indicative of the recognizable text content 664. The predeterminedcondition may include a gesture with the finger and/or fingertipproximate to the recognizable text content. For example, in anembodiment, the predetermined condition may include tapping thefingertip three times on the page surface 661 proximate to therecognizable text content. In another embodiment, the predeterminedcondition may include the user activating the switch 644. In a furtherembodiment, the predetermined condition may include the user speaking avoice command received at the microphone 622. In an embodiment, thepredetermined condition may include the fingertip remaining stationaryand proximate to the recognizable text content for a predetermined time.The context-detector generates information indicative of a recognizableaspect of the text content in response to the acquired data.Alternatively, the fingertip may be placed proximate to anotherrecognizable aspect of the book 660. For example, another recognizableaspect of the book may include the cover title 684.

The user may create an expression associated with the recognizableaspect of the book 660. In an embodiment, the user may create a verbalexpression for association with the recognizable aspect of the item. Theverbal expression may be received by the microphone 622. In anotherembodiment, the user may create the expression by a user gestureassociated with the recognizable aspect of the item. The user gesturemay be formed at least a part by a user limb associated with therecognizable aspect of the item. The user gesture may be captured by theimage capture module 646.

In another embodiment, the annotation environment capture device mayremain incorporated in the system 612, and a microphone and anannotating device may be incorporated into a structure of the book 660.For example, the microphone and annotating device may be incorporated asillustrated in conjunction with the book 661 of FIG. 15.

FIG. 17 illustrates an environment 800 that includes a partial view ofan exemplary annotation system and a partial view of an exemplary item.The exemplary annotation system is illustrated as an exemplary humanwearable system 610, which is further illustrated as the exemplary headmountable system 612. The exemplary item is illustrated as an exemplarybound paper book 760. The exemplary book illustrates several exemplaryrecognizable aspects, including a cover 782, an author name 786,illustrated as Homer, a cover title 784, illustrated as THE ILIAD, andan electronically readable product code 788, illustrated as a bar code.Further illustrated exemplary recognizable aspects include a page havinga page number 766, illustrated as page number “69,” a page surface 761,and a binding 780 having a binding title 783, illustrated as THE ILIAD.The exemplary book also illustrates an exemplary recognizable aspectthat includes a recognizable text content 664 printed on and native tothe page surface 761 of the page number 69.

FIG. 17 also includes a handwriting instrument, illustrated as a pen 720having a writing tip 721. The pen is illustrated as having formed thehandwritten characters “Where is Troy?” 750 in response to a useraction.

The exemplary head mountable system 612 includes the frame 614 asdescribed in conjunction with FIG. 13. The frame includes ahandwriting-tracking module 744 operable to acquire data indicative ofthe handwritten annotation 750. In an embodiment, thehandwriting-tracking module 744 includes operability to acquire dataindicative of the handwritten annotation in response to detectedmovements by the pen-writing tip 721. In another embodiment, thehandwriting-tracking module includes operability acquire data indicativeof the handwritten annotation in response to images of the pen-writingtip and/or the handwritten characters. The images may be captured by thecapture module 646.

In operation of an embodiment, a user may don and use the exemplary headmountable system 612, and allow the system to generate informationindicative of a recognizable aspect of an item in a manner at leastsubstantially similar to the system 612 of FIG. 13. In addition, theuser may grasp the pen 720 in their hand and hand generate a userexpression, such as the annotation 750, using the pen tip 721 andproximate to a recognizable aspect, such as the recognizable textcontent 664. Data indicative of the hand-generated annotation isacquired by the handwriting-tracking module 744, and the annotatingdevice 640 generates information indicative of the user expressionassociated with the recognizable text content 664.

In an alternative embodiment not illustrated, the handwriting instrumentmay include an annotating device operable to generate informationindicative of a user expression associated with the recognizable aspectof the item. The annotating device may be at least substantially similarto the annotating device 530 described in conjunction with FIG. 12.

FIG. 18 illustrates an environment 850 that includes a partial view ofan exemplary annotation system and a partial view of an exemplary item.The exemplary annotation system is illustrated as an exemplary humanwearable system 610, which is further illustrated as an exemplary headmountable system 612 of FIG. 13 and others. The exemplary item isillustrated as an exemplary document with a page 866 thereof displayedthrough a surface 192 of the monitor 191 of the computing systemenvironment 100 of FIG. 2. The displayed page includes a recognizabletext content 868. The head mountable system 612 includes astylus-tracking module 745 operable to acquire data indicative of apreformed annotation.

The environment 850 also includes a stylus corresponding to a preformeduser expression. In an embodiment illustrated in FIG. 18, the stylus isillustrated as a stylus set 860 of three individual styluses thatinclude a green stylus 861, a blue stylus 862, and a yellow stylus 863.In an embodiment, the set of styluses are illustrative of “hi-liter”type highlighting markers used by students and others to annotate coursematerials. A stylus may have any preformed expression. For example, inan embodiment, the yellow stylus may correspond to a preformed userexpression of “highly interesting material.” the red stylus maycorrespond to a preformed user expression of “dumb material.” In anembodiment, a stylus may transmit a signal indicative of a preselectedor preformed annotation. In another embodiment, a stylus may include anoptically recognizable feature indicative of a preselected or preformedannotation. An optically recognizable feature may include a color, a barcode, and/or a shape.

In operation of an embodiment, a user may don and use the exemplary headmountable system 612, and allow the system to generate informationindicative of a recognizable aspect of an item in a manner at leastsubstantially similar to the system 612 of FIG. 13. In addition, theuser may grasp a stylus in their hand, such as the yellow stylus 863,and use the stylus to associate a preformed user expression with therecognizable text content 868 of the document. The user expression maybe associated in any manner that may be captured by the stylus-trackingmodule 745. For example, in an embodiment, the stylus-tracking modulemay be operable to capture a tapping of a tip of the yellow stylus 863proximate to the recognizable text content as a preformed userexpression of “highly interesting material.” In another embodiment, thestylus-tracking module may be operable to capture a closed figure drawnaround the recognizable text content with the tip of a stylus as apreformed user expression, the nature of which is established by theparticular stylus selected from the stylus set 860. The annotatingdevice 630 generates information indicative of the user expressionassociated with the recognizable text context in response to thecaptured preformed user expression.

FIG. 19 illustrates an exemplary operational flow 900. After a startoperation, the operational flow moves to a context operation 910. At thecontext operation, a first information indicative of a recognizableaspect of an item is generated in a portable apparatus. At a contextoperation 920, a second information indicative of a user expressionassociated with the recognizable aspect of an item is generated in theportable apparatus. The operational flow then moves to an end operation.

FIG. 20 illustrates an alternative embodiment of the exemplaryoperational flow 900 of FIG. 19. The context operation 910 may includeat least one additional operation, such as an operation 912. At theoperation 912, a first information indicative of a recognizable aspectof an item is generated in a portable head mountable apparatus. Thecontent operation 920 may include at least one additional operation,such as an operation 922. At the operation 922, a second informationindicative of a user expression associated with the recognizable aspectof an item is generated in the portable head mountable apparatus.

FIG. 21 illustrates another alternative embodiment of the exemplaryoperational flow 900 of FIG. 19. The operational flow 900 may include atleast one additional operation 930. The additional operation 930 mayinclude a storage operation 932 and/or a communication operation 934.The storage operation 932 may include saving at least one of the firstinformation indicative of a recognizable aspect of an item and thesecond information indicative of a user expression associated with therecognizable aspect of an item. The communication operation 934 mayinclude transmitting least one of the first information indicative of arecognizable aspect of an item and the second information indicative ofa user expression associated with the recognizable aspect of an item.

FIG. 22 illustrates a partial view of an exemplary wearable apparatus950. The apparatus includes a context module 955 operable to generate afirst information indicative of a recognizable aspect of an item. Theapparatus also includes an annotation module 960 operable to generate asecond information indicative of a user expression associated with therecognizable aspect of an item. In an embodiment, the apparatus mayinclude at least one additional module. The additional module mayinclude a communications module 965 operable to transmit at least one ofthe first information indicative of a recognizable aspect of an item andthe second information indicative of a user expression associated withthe recognizable aspect of an item. The additional module may include astorage module 970 operable to save at least one of the firstinformation indicative of a recognizable aspect of an item and thesecond information indicative of a user expression associated with therecognizable aspect of an item.

FIG. 23 illustrates an exemplary operational flow 1000. After a startoperation, the operational flow moves to a monitoring operation 1005. Atthe monitoring operation, a gaze of a user's eye is tracked. At acapture operation 1010, an image is acquired of a recognizable aspect ofan item corresponding to the gaze of a user's eye. At a contextprocessing operation 1015, a first information is generated indicativeof the recognizable aspect of an item corresponding to the gaze of auser's eye. At an annotation processing operation 1020, a secondinformation is generated indicative of a user expression associated withthe recognizable aspect of an item corresponding to the gaze of a user'seye. The operational flow then proceeds to an end operation.

In an embodiment, the operational flow 1000 may include at least oneadditional operation, such as an operation 1030. At the operation 1030,a signal is received indicative of the user expression associated withthe recognizable aspect of an item.

FIG. 24 illustrates an alternative embodiment of the operational flow1000 of FIG. 23. In an embodiment, the operational flow 1000 may includean operational flow 1032 performed in a head mountable apparatus. In afurther embodiment, the capture operation 1010 may include at least oneadditional operation, such as an operation 1012. At the operation 1012,an image is acquired of a recognizable aspect of an item correspondingto the gaze of a user's eye in response to a received user input. Thereceived user command may include any suitable command, for example asound, such as a finger snap, a voice command, such as “acquire this,”and/or activation of a button.

FIG. 25 illustrates another alternative embodiment of the operationalflow 1000 of FIG. 23. The annotation processing operation 1020 mayinclude at least one additional operation. An additional operation mayinclude an operation 1022, an operation 1024, and an operation 1026. Atthe operation 1022, a second information is generated indicative of auser verbal expression associated with the recognizable aspect of anitem. At the operation 1024, a second information is generatedindicative of a user hand formed expression associated with therecognizable aspect of an item. At the operation 1026 a secondinformation is generated indicative of a user gesture associated withthe recognizable aspect of an item.

FIG. 26 illustrates a partial view of an exemplary wearable apparatus1050. The apparatus includes a monitoring module 1055 operable to tracka gaze of a user's eye, and a capture module 1060 operable to acquire animage of a recognizable aspect of an item corresponding to the gaze of auser's eye. The apparatus also includes a context module 1065 operableto generate a first information indicative of the recognizable aspect ofan item corresponding to the gaze of a user's eye. The apparatusincludes an annotation module 1070 operable to generate a secondinformation indicative of a user expression associated with therecognizable aspect of an item. In an alternative embodiment, theapparatus may include a receiver module 1075 operable to receive asignal indicative of the user expression associated with therecognizable aspect of an item. The signal indicative of the userexpression may include a video signal indicative of a handwritten userexpression, a signal indicative of a spoken user expression, and/or asignal indicative of a user gesture (none shown).

FIG. 27 illustrates an environment 1900 in which embodiments may beimplemented. The environment includes an article of manufactureincluding a surface, illustrated as a document 1910 including a surface1912. The environment also includes a tag characterization tableincluding at least two machine-distinguishable tags, illustrated as acharacterization table 1920 including at least twomachine-distinguishable tags 1928 displayed in a column. The tagcharacterization table further has a unique descriptor respectivelycorrelating to each machine-distinguishable tag of the at least twomachine-distinguishable tags, illustrated as a column 1930 of uniquedescriptors. In an embodiment, a machine-distinguishable tag 1924 isillustrated as a “●” and is correlated to a unique descriptor “urgenttasks” 1932. In an embodiment, the tag characterization table may bedisplayed on a separate article of manufacture. In another embodiment,the tag characterization table may be printed on the document 1910,displayed on an electronic display surface (not shown), and/or known toa user.

The environment also includes an exemplary handheld marking device, inan embodiment illustrated as pen having some or all of the elements ofany one or all of the exemplary handheld marking device 210 of FIG. 3,the exemplary electronic pen 211 of FIG. 4, the exemplary apparatus 212of FIG. 5, the exemplary apparatus 213 of FIG. 6, and/or the exemplaryhandheld system 510 of FIG. 21. In another embodiment, the exemplaryhandheld marking device may include the pen 720 having a writing tip 721and the head-mountable system 612 of FIG. 17. The exemplary handheldmarking device includes a writing detector module operable to generate asignal indicative of a hand-formed mark by a writing element on asurface, and a tag detector module operable to generate a signalindicative of a machine-distinguishable tag associated with thehand-formed mark. In an embodiment, the writing detector module isillustrated as the writing detector module 230 of the exemplary handheldwriting device 210. The tag detector module is further illustrated asthe context detector module 240.

In an alternative embodiment (not shown), the environment 1900 mayinclude a device that generates movement tracking data corresponding tomovements of the handheld marking device and/or the article ofmanufacture. For example, Polhemus of Colchester, Vt., manufactures andmarkets several electromagnetic tracking apparatus that tracks objectsin a space. Movement tracking data may be used in generating a contentsignal indicative of a hand-formed mark on a surface, and/or generatinga label signal indicative of a machine-distinguishable tag associatedwith the hand-formed mark.

In an embodiment, the surface 1912 of document 1910 is initially blank.In use, and as illustrated in FIG. 27, a user moves the writing element220 of the handheld writing device 210 to form at least one instance ofa hand-formed mark 1944 on the surface 1912. FIG. 27 illustratesinstances of hand-formed marks each respectively formed on a singleline. The user also hand-forms a machine-distinguishable tag on thesurface 1912 and respectively associates the machine-distinguishable tagwith at least one hand-formed mark. The machine-distinguishable tag isselected from the at least two machine-distinguishable tags 1928. Forexample, FIG. 27 illustrates a hand-formed machine-distinguishable tag“●” 1914 proximally associated with a hand-formed mark, illustrated as atextual entry “Buy welcome home card” 1916. The exemplary handheldmarking device captures the hand-formed mark and generates a contentsignal indicative of the hand-formed mark using the writing detectormodule 230. The exemplary handheld marking device also captures thehand-formed machine-distinguishable tag using the writing detectormodule 230, and/or the context detector module 240, and generates alabel signal indicative of the hand-formed machine-distinguishable tag.

FIG. 28 illustrates another environment 1980 in which embodiments may beimplemented. The environment includes an article of manufacture,illustrated as a document 1911 including a surface 1912. The surfaceincludes a writing surface portion 1916 and a tag portion 1918. Thewriting surface portion accepts at least one mark by a handheld markingdevice. The tag portion displays the at least twomachine-distinguishable tags 1928, each machine-distinguishable tag ofthe at least two machine distinguishable tags is unique and userassociatable with at least one mark. A tag may be associated by using ahandheld marking device, such as the device 210-213, 510, and/or 720.

In an embodiment, the writing surface portion 1916 is initially blank.In use, and as illustrated in FIG. 28, a user moves the handheld markingdevice to form at least one hand-formed mark 1944 on the surface 1912 ofthe handwriting portion. The user also associates amachine-distinguishable tag with the at least one hand-formed mark. Inan embodiment illustrated in FIG. 28, a user associates amachine-distinguishable tag with at least one hand-formed mark bycapturing the machine-distinguishable tag from the at least twomachine-distinguishable tags of the tag portion 1918. In an embodiment,the machine-distinguishable tag may be captured from the tag portion asan image using the image capture device 246 of the context detectormodule 240. A user may associate the machine-distinguishable tag “●”1924 of the tags 1928 with the hand-formed mark “Buy welcome home card”1916 by capturing an image (represented as the captured image 1915) ofthe machine-distinguishable tag “●” with the context detector module240. The association may include a temporal association between formingthe hand-formed mark 1916 and capturing the machine-distinguishable tag“●”. In a further embodiment, the association may include a gestureindicating an association between the hand-formed mark 1916 andcapturing the machine-distinguishable tag “●”. For example, a gesturemay include tapping a portion of the handheld marking device on thesurface 1912, and/or tapping a fingertip on any surface. In anotherembodiment, a scanner (not shown) embodied in the handheld markingdevice may be used to discern the machine-distinguishable tag “●” forassociation with the hand-formed mark 1916.

FIG. 29 illustrates an exemplary operational flow 1700. After a startoperation, the flow moves to a mark description operation 1710. At themark description operation 1710, a content signal indicative of ahand-formed mark on a surface is generated. At a description operation1740, a label signal is generated indicative of amachine-distinguishable tag associated with the hand-formed mark. Themachine-distinguishable tag is selected from at least twomachine-distinguishable tags. At a dissemination operation 1780, arepresentation of the hand-formed mark is distributed to a datarecipient in response to the machine-distinguishable tag. In anembodiment, a data recipient may include at least one of an application,a program, a file, an item list file, an automation, and/or a storage.The operational flow then moves to an end operation.

Operation of an embodiment of the exemplary operational flow 1700 may beillustrated with reference to FIG. 27. A user moves the writing element220 to hand form the mark “Buy welcome home card” 1916 on the surface1912. The operation 1710 generates a content signal indicative of thehand-formed mark 1916. The user moves the writing element 220 to handform the machine-distinguishable tag 1914 and associate the tag with thehand-formed mark. The operation 1740 generates a label signal indicativeof the machine-distinguishable tag 1914 associated with the hand-formedmark 1916. The operation 1780 distributes a representation of thehand-formed mark to a data recipient in response to themachine-distinguishable tag. The data recipient may be associated withthe handheld writing apparatus, or may be associated with anothercomputing device, for example, such as the thin computing device 20 ofFIG. 1, and/or the general-purpose computing device 110 of FIG. 2.

FIG. 30 illustrates an alternative embodiment of the exemplaryoperational flow 1700 of FIG. 29. The mark description operation 1710may include at least one additional operation. An additional operationmay include operation 1712, an operation 1714, and operation 1732, anoperation 1734, and/or an operation 1736. The operation 1712 generates acontent signal in a handheld writing device indicative of a hand-formedmark on a surface. The handheld writing device may include the exemplaryhandheld marking device of FIG. 27. The operation 1714 generates acontent signal in a handheld scanning device indicative of a hand-formedmark on a surface. The operation 1732 generates a content signalindicative of a handwriting on a surface. The operation 1734 generates acontent signal indicative of a hand drawn figure on a surface. At theoperation 1736, the hand-formed mark includes at least one of a pattern,a shade, a tone, a letter, a word, a number, an alphanumeric, acharacter, an icon, a geometric shape, a figure, a graphic, a glyph, arepeated pattern, a random pattern, an image, a region, and/or a legend.

FIG. 31 illustrates another alternative embodiment of the exemplaryoperational flow 1700 of FIG. 29. The mark operation 1710 may include atleast one additional operation, such as an operation 1718. The operation1718 generates a content signal indicative of a handwritten mark on asurface. The operation 1718 may include at least one additionaloperation, such as an operation 1720. The operation 1720 generates acontent signal in a handheld device indicative of a handwritten mark ona surface. The operation 1720 may include at least one additionaloperation, such as an operation 1722. The operation 1722 generates acontent signal in a handheld device indicative of a mark handwritten ona surface by a writing element of the handheld device.

FIG. 32 illustrates a further alternative embodiment of the exemplaryoperational flow 1700 of FIG. 29. The mark operation 1710 may include atleast one additional operation. An additional operation may include anoperation 1724, an operation 1728, and/or an operation 1730. Theoperation 1724 generates a content signal indicative of a user handdrawn mark on a surface. The operation 1724 may include at least oneadditional operation, such as an operation 1726. The operation 1726generates a content signal in a handheld device indicative of a usermark hand drawn on a surface by a writing element of the handhelddevice. The operation 1728 generates a content signal indicative of auser hand-formed and machine-distinguishable mark on a surface. Theoperation 1730 generates a content signal indicative of a hand-formedand human recognizable mark on a surface.

FIG. 33 illustrates an alternative embodiment of the exemplaryoperational flow 1700 of FIG. 29. The description operation 1740 mayinclude at least one additional operation. Additional operations mayinclude an operation 1742, an operation 1744, an operation 1746, anoperation 1750, and/or an operation 1752. The operation 1742 generates alabel signal indicative of a hand-formed machine-distinguishable tagassociated with the hand-formed mark. The operation 1744 generates alabel signal indicative of a hand-formed machine-distinguishable tagassociated by a user with the hand-formed mark. The operation 1746generates a label signal indicative of a machine-distinguishable tagpreformed on the surface and associated with the hand-formed mark. Theoperation 1746 may include at least one additional operation, such as anoperation 1748. The operation 1748 generates a label signal indicativeof a machine-distinguishable tag preformed on the surface and associatedby a user with the hand-formed mark. The operation 1750 generates alabel signal indicative of a machine-distinguishable tag associated withthe hand-formed mark in a handheld device that generated the contentsignal indicative of a hand-formed mark. The operation 1752 generates alabel signal indicative of a machine-distinguishable tag associated by auser with the hand-formed mark in a handheld device that generated thecontent signal indicative of a hand-formed mark.

FIG. 34 illustrates another alternative embodiment of the exemplaryoperational flow 1700 of FIG. 29. The description operation 1740 mayinclude at least one additional operation. Additional operations mayinclude an operation 1754, an operation 1756, an operation 1758, anoperation 1764, and/or an operation 1770. The operation 1754 generates alabel signal indicative of a machine-distinguishable tag proximallyassociated with the hand-formed mark. The operation 1756 generates alabel signal indicative of a machine-distinguishable tag proximallyassociated by a user with the hand-formed mark. The operation 1758generates a label signal indicative of a machine-distinguishable taggesturally associated by a user with the hand-formed mark. In anembodiment, a gestural association may include a movement of a writingelement of a handheld writing device in contact with a surface, such asa movement across a surface by the writing element 220 of the handheldwriting device 210 of FIG. 3. In another embodiment, a gesturalassociation may include a movement away from a surface by a writingelement of a handheld writing device, such as an air movement of thewriting element 220 of the handheld writing device 210 of FIG. 3. In afurther embodiment, a gestural association may include a movement of oneor more body parts, such as a tapping movement of the finger 710 of FIG.16 with respect to a machine-recognizable tag and/or with respect to ahand-formed mark. In another embodiment, a machine-recognizable tag mayinclude a gesture by at least one body part.

At the operation 1764, the machine-distinguishable tag includes at leastone character. At the operation 1770, the machine-distinguishable tagincludes at least one hand-formed machine-distinguishable tag selectedfrom the following: a pattern, a shade, a tone, a letter, a word, anumber, an alphanumeric, a character, an icon, a geometric shape, afigure, a graphic, a glyph, a repeated pattern, a random pattern, animage, a region, and/or a legend.

FIG. 35 illustrates a further alternative embodiment of the exemplaryoperational flow 1700 of FIG. 29. The description operation 1740 mayinclude at least one additional operation. Additional operations mayinclude an operation 1760, an operation 1762, an operation 1766, and/oran operation 1768. At the operation 1760, the machine-distinguishabletag is selected from at least two machine-distinguishable user definedtags. The operation 1760 may include at least one additional operation,such as an operation 1761. At the operation 1761, themachine-distinguishable tag is selected from a tag characterizationtable having at least two machine-distinguishable tags and a uniqueuser-created descriptor respectively correlating to each tag. At theoperation 1762, the machine-distinguishable tag is selected from atleast two machine-distinguishable predefined tags. The operation 1762may include at least one additional operation, such as an operation1763. At the operation 1763, the machine-distinguishable tag is selectedfrom a tag characterization table having at least twomachine-distinguishable tags and a unique predefined descriptorrespectively correlating to each tag. At the operation 1766, themachine-distinguishable tag includes at least one character representingan information indicative of the hand-formed mark. At the operation1768, the machine-distinguishable tag includes at least one characterrepresenting an identification of the hand-formed mark.

FIG. 36 illustrates a further alternative embodiment of the exemplaryoperational flow 1700 of FIG. 29. The dissemination operation 1780 mayinclude at least one additional operation. Additional operations mayinclude an operation 1782, an operation 1784, an operation 1786, and/oran operation 1788. The operation 1782 distributes a representation ofthe content signal indicative of the hand-formed mark to a datarecipient in response to the label signal. The operation 1784distributes a representation of the hand-formed mark in response to themachine-distinguishable tag to a data recipient associated with ahandheld device that generated the content signal indicative of ahand-formed mark. The operation 1786 distributes a representation of thehand-formed mark to a data recipient associated with an application inresponse to the machine-distinguishable tag. The operation 1788distributes a representation of the hand-formed mark to a data recipientassociated with an application in response to a lookup table.

In an embodiment, the distributes a representation of the hand-formedmark to a data recipient associated with an application in response to alookup table may include a distributes a representation of thehand-formed mark to a data recipient associated with an application byemploying a lookup table. In another embodiment, the distributes arepresentation of the hand-formed mark to a data recipient associatedwith an application in response to a lookup table may include adistributes a representation of the hand-formed mark to a data recipientassociated with an application guided by a lookup table. In a furtherembodiment, the distributes a representation of the hand-formed mark toa data recipient associated with an application in response to a lookuptable may include a distributes a representation of the hand-formed markto a data recipient associated with an application directed by a lookuptable.

The operation 1788 may include at least one additional operation. Anadditional operation may include an operation 1790, and/or an operation1792. The operation 1790 distributes a representation of the hand-formedmark to a data recipient associated with an application in response to auser-configurable lookup table. The operation 1792 distributes arepresentation of the hand-formed mark to a data recipient associatedwith an application in response to a lookup table includingpredetermined values.

FIG. 37 illustrates an alternative embodiment of the exemplaryoperational flow 1700 of FIG. 29. The dissemination operation 1780 mayinclude at least one additional operation. Additional operations mayinclude an operation 1794, an operation 1796, an operation 1798, anoperation 1802, and/or an operation 1804. The operation 1794 distributesa representation of the hand-formed mark to a data recipient in responseto a received user action. The operation 1796 distributes arepresentation of the hand-formed mark to a retrievable storage file inresponse to the machine-distinguishable tag. The operation 1798distributes a representation of the hand-formed mark to a data recipientin response to the label signal indicative of a machine-distinguishabletag. The operation 1802 distributes a recognized representation of thehand-formed mark to a data recipient in response to themachine-distinguishable tag. The operation 1804 distributes the contentsignal indicative of the hand-formed mark to a data recipient inresponse to the machine-distinguishable tag.

FIG. 38 illustrates another alternative embodiment of the exemplaryoperational flow 1700 of FIG. 29. The dissemination operation 1780 mayinclude at least one additional operation. Additional operations mayinclude an operation 1805, an operation 1806, an operation 1807, and/oran operation 1808. The operation 1805 distributes a representation ofthe hand-formed mark to an item list file in response to themachine-distinguishable tag. The operation 1806 distributes arepresentation of the hand-formed mark to an application in response tothe machine-distinguishable tag. The operation 1807 distributes arepresentation of the hand-formed mark to a program in response to themachine-distinguishable tag. The operation 1808 exposes a representationof the hand-formed mark to an automation client in response to themachine-distinguishable tag. In an embodiment, an automation may includea particular usage of a component software architecture that letsapplications expose their internal functions as objects to otherapplications. “Automation” includes “OLE automation.”

FIG. 39 illustrates a further alternative embodiment of the exemplaryoperational flow 1700 of FIG. 29, which includes additional operations.Another mark representation operation 1812 generates another contentsignal indicative of another hand-formed mark on the surface. Anotherdescription operation 1814 generates another label signal indicative ofanother machine-distinguishable tag associated with the anotherhand-formed mark. The another machine-distinguishable tag is selectedfrom the plurality of machine-distinguishable tags. Anotherdissemination operation 1816 distributes a representation of the anotherhand-formed mark to another data recipient in response to the anothermachine-distinguishable tag.

FIG. 40 illustrates a partial view of an exemplary apparatus 1820. Theapparatus includes a means 1832 for generating a content signalindicative of a hand-formed mark on a surface. The apparatus alsoincludes a means 1834 for generating a label signal indicative of amachine-distinguishable tag associated with the hand-formed mark. Theapparatus further includes a means 1836 for distributing arepresentation of the hand-formed mark to an item file in response tothe label signal indicative of a machine-distinguishable tag. Analternative embodiment may provide at least one additional means, suchas a means 1842. The means 1842 includes a first handheld means thatincludes the means 1832 for generating a content signal indicative of ahand-formed mark on a surface, and the means 1834 for generating a labelsignal indicative of a machine-distinguishable tag associated with thehand-formed mark. The means 1842 also includes a second means thatincludes the means 1836 for distributing a representation of thehand-formed mark to an item file in response to the label signalindicative of a machine-distinguishable tag.

FIG. 41 illustrates an exemplary operational flow 1850. After a startoperation, the operation flow moves to a mark representation operation1852. The operation 1852 generates a content signal indicative of ahand-formed mark. The operation 1854 generates a label signal indicativeof a machine-distinguishable identifier associated with the hand-formedmark. The machine-distinguishable identifier is one of at least twomachine-distinguishable identifiers. The operation 1856, in response toa user action, transmits the content signal indicative of a hand-formedmark and the label signal indicative of a machine-distinguishableidentifier associated with the hand-formed mark. The operational flowthen proceeds to an end operation.

In an alternative embodiment, the operational flow 1850 may include anadditional operation, such as an operation 1860. The operation 1860includes an operation 1862, an operation 1864, and an operation 1866.The operation 1862 receives the label signal indicative of amachine-distinguishable identifier associated with the hand-formed mark.The operation 1864 receives the content signal indicative of ahand-formed mark. The operation 1866 distributes a representation of thehand-formed mark to a data recipient in response to the label signal.

This description returns to FIG. 27 and the environment 1900 in whichembodiment may be implemented. The environment includes an article ofmanufacture, illustrated as the document 1910 including a surface 1912.The document includes a handwritten mark 1916 on the surface 1912 formedby a pen. The document includes a machine-distinguishable tag 1914associated by a user with the handwritten mark. Themachine-distinguishable tag being selected by the user from at least twomachine-distinguishable tags 1928.

The environment includes a pen, an embodiment of which is illustrated asthe handheld writing device 210 of FIG. 3. The pen includes a writingelement, illustrated in an embodiment as the writing element 220. Thepen also includes a writing detector module operable to generate asignal indicative of the handwritten mark by the writing element on thesurface. The writing detector module is illustrated in an embodiment asthe writing detector module 230. The pen includes a tag detector moduleoperable to generate a signal indicative of the machine-distinguishabletag associated with the handwritten mark. The tag detector module isillustrated as the context detector module 240. In another embodiment,the writing detector module includes the context detector module. In afurther embodiment, the machine-distinguishable tag associated with thehandwritten mark includes a machine-distinguishable tag hand marked onthe surface and associated with the handwritten mark.

In an embodiment, the environment 1900 includes a computer programproduct, such as the computer program product 2150 of FIG. 47. Thecomputer program product includes program instructions operable toperform a process in a computer system. The process includes receiving asignal indicative of the handwritten mark. The process also includesreceiving a signal indicative of the machine-distinguishable tagassociated by the user with the handwritten mark, themachine-distinguishable tag being selected from at least twomachine-distinguishable tags. The process further includes distributinga representation of the handwritten mark to a data recipient in responseto the machine-distinguishable tag. The computer program productincludes a computer-readable signal-bearing medium bearing the programinstructions.

FIG. 42 illustrates an exemplary operational flow 2000. After a startoperation, the operation flow moves to a mark representation operation2010. The mark representation operation 2010 generates a content signalindicative of a hand-formed mark. The description operation 2020generates a label signal indicative of a distinct identifier associatedwith the hand-formed mark. The distinct identifier includes one of atleast two distinct identifiers. The distribution operation 2030 performsan action with respect to a representation of the hand-formed mark inresponse to the distinct identifier. The operational flow then proceedsto an end operation.

FIG. 43 illustrates an alternative embodiment of the operational flow2000 of FIG. 42. The description operation 2020 may include at least oneadditional operation, such as an operation 2022. At the operation 2022,a distinct identifier is selected from at least two distinct identifierseach respectively keyed to a unique data recipient. The operation 2022may include at least one additional operation. An additional operationmay include an operation 2024, an operation 2026, and/or an operation2028. At the operation 2024, a distinct identifier is selected from atleast two distinct identifiers each respectively corresponding to aunique data recipient. At the operation 2026, a distinct identifier isselected from at least two distinct identifiers each respectivelyidentifying a unique data recipient. At the operation 2028, a distinctidentifier is keyed to at least one of a file, a record, a database,and/or a storage medium.

FIG. 44 illustrates an alternative embodiment of the operational flow2000 of FIG. 42. The distribution operation 2030 may include at leastone additional operation. An additional operation may include anoperation 2032, an operation 2036, an operation 2038, and/or anoperation 2040. The operation 2032 distributes a representation of thehand-formed mark to a data recipient in response to the distinctidentifier. The operation 2032 may include at least one additionaloperation, such as an operation 2034. At the operation 2034, the datarecipient includes at least one of an application, a program, a file, arecord, a database, a storage medium, and/or another method. Theoperation 2036 associates a representation of the hand-formed mark witha data recipient in response to the distinct identifier. The operation2038 processes a representation of the hand-formed mark in response tothe distinct identifier. The operation 2040 exposes a representation ofthe hand-formed mark to an automation in response to the distinctidentifier.

FIG. 45 illustrates an exemplary operational flow 2100. After a startoperation, the operation flow moves to a mark reception operation 2110.The mark reception operation 2110 receives from a handheld writingdevice a signal indicative of a handwriting on a surface. The descriptorreception operation 2120 receives a signal indicative of amachine-distinguishable tag associated by a user with the handwriting.The machine-distinguishable tag is selected from at least twomachine-distinguishable defined tags. The delivery operation 2130distributes a representation of the handwriting to a data recipient inresponse to the machine-distinguishable tag. The operational flow thenmoves to an end operation.

In an embodiment, the mark reception operation 2110 may include at leastone additional operation, such as an operation 2112. The operation 2112receives a signal indicative of a handwriting on a paper surface. Inanother embodiment, the descriptor reception operation 2120 may includeat least one additional operation. An additional operation may includean operation 2124, an operation 2126, and/or an operation 2128. Theoperation 2124 recognizes the machine-distinguishable tag associated bythe user with the handwriting. The operation 2126 characterizes themachine-distinguishable tag associated by the user with the handwriting.The operation 2128 receives a signal originated by the handheld writingdevice and indicative of a machine-distinguishable tag associated by theuser with the handwriting.

FIG. 46 illustrates an alternative embodiment of the exemplaryoperational flow 2100. The operational flow includes another markreception operation 2112 that receives another signal originated by thehandheld writing device and indicative of another handwriting on thesurface. The operational flow includes another description receptionoperation 2122 that receives another signal indicative of anothermachine-distinguishable tag associated by the user with the anotherhandwriting. The machine-distinguishable tag being selected from atleast two machine-distinguishable defined tags. The operational flowincludes another delivery operation 2132 distributes a representation ofthe another handwriting to another data recipient associated with thecomputing device in response to the another machine-distinguishable tag.

FIG. 47 illustrates a partial view of an exemplary computer programproduct 2150 that includes a computer program 2154 and acomputer-readable signal bearing medium 2152 bearing the computerprogram. The computer program product encodes the computer program ascomputer executable instructions for executing a computer process on acomputing device. The computer program includes a computer process thatreceives a signal indicative of a handwriting. The computer process alsoreceives a signal indicative of a machine-distinguishable tag associatedby a user with the handwriting, the machine-distinguishable tag beingselected from at least two machine-distinguishable tags. The computerprocess further distributes a representation of the handwriting to adata recipient in response to the machine-distinguishable tag.

In an alternative embodiment, the computer process 2154 may furtherinclude an additional process, such as a process 2156, and/or a process2158. The process 2156 distributes a representation of the handwritingto a data recipient saved in a computer-readable storage mediumassociated with the computing system in response to themachine-distinguishable tag. The process 2158 distributes arepresentation of the signal indicative of a handwriting to a datarecipient saved in a computer-readable storage medium associated withthe computing system in response to the signal indicative of amachine-distinguishable tag.

In another embodiment, the computer-readable signal-bearing medium 2152may include a computer storage medium 2160, which may be carried by acomputer-readable carrier (not shown). The computer-readablesignal-bearing medium may include a communications medium 2162. In afurther alternative embodiment, the computer-program 2154 may beimplemented in hardware, software, and/or firmware.

FIG. 48 illustrates an exemplary system 2200 in which embodiments may beimplemented. The system includes an exemplary handheld marking device,illustrated in an embodiment as the exemplary handheld marking devicedescribed in conjunction with FIG. 27. The system includes an exemplarycomputing device, illustrated in an embodiment as the thin computingdevice 20 described in conjunction with FIG. 1, and/or thegeneral-purpose computing device 110 described in conjunction with FIG.2. The system further includes a handwriting distribution program 2210that includes a tag distribution table 2220. The distribution program2210 is at least substantially similar to the computer program 2154 ofthe computer-program product 2150 described in conjunction with FIG. 47.In an embodiment, the handwriting distribution program is saved in thenon-removable non-volatile memory hard disk drive 141 of thegeneral-purpose computing device.

FIG. 48 further illustrates an embodiment where a data recipientincludes an item list file 2230 saved in a computer storage media, suchas the system memory 130 and/or a computer storage media productassociated with the computing device, such as the non-removablenon-volatile memory hard drive 141. The item list file includes at leasttwo list subfiles, one of which is illustrated as an “Urgent task list”subfile 2232. FIG. 48 further illustrates an embodiment where the tagdistribution table correlates a machine-distinguishable tag with a listfile of at least two item list files. For example, the tag distributiontable correlates the machine-distinguishable tag “●” 1924 and anindicated subfile 2222, illustrated as the “Urgent task list” subfile.In another embodiment, a data recipient may include a file associatedwith application programs 145 and/or program data 147 saved on thegeneral-purpose computing device 110. For example, themachine-distinguishable tag “●” may be correlated with a subfileassociated with Microsoft's Outlook program, such as its Tasks folder.

In operation of an embodiment, a content signal indicative of ahand-formed mark by a writing element of a handheld marking device iscommunicated between the computing device 110 and the handheld markingdevice, such as the handheld marking device of FIG. 27. In anembodiment, the content signal includes the content signal 1710indicative of a hand-formed mark on a surface as described inconjunction with FIG. 29. The content signal may be communicated over acommunications link 2243 that may include a wired coupler (not shown) orover a wireless coupling. Further, a label signal 2242 indicative of amachine-distinguishable tag associated with the hand-formed mark iscommunicated between the computing device 110 and the handheld markingdevice, such as the handheld marking device of FIG. 27. In anembodiment, the label signal includes the label signal indicative of amachine-distinguishable tag associated with the hand-formed mark 1740described in conjunction with FIG. 29. The label signal may becommunicated over the communication link 2243 as illustrated in FIG. 48.

In an alternative embodiment, a scanner 2250 may be acquire andcommunicate the content signal 2240 and/or the label signal 2242 to thecomputing device 110. The communication link 2245 between the scannerand the computing device may include a wireless coupling and/or a wiredcoupling (not shown). In another embodiment, a communication path of thecontent signal and/or the label signal between the handheld markingdevice and the computing device may include an intermediate device 2252.The intermediate device may include a portable computing device, such asa personal digital assistant (PDA), a cell phone, and/or a laptopcomputing device. The intermediate device receives the content signal2240 and/or the label signal 2242 from the handheld marking device overa communications link 2247. The communication link 2247 between theintermediate device and the handheld marking device may include awireless coupling and/or a wired coupling (not shown). The intermediatedevice transmits the content signal 2240 and/or the label signal 2242 tothe computing device over a communications link 2249. The intermediatedevice may perform at least one processing step on the content signal2240 and/or the label signal 2242 before communicating those signals tothe computing device.

Continuing with the operation of an embodiment illustrated in FIG. 48,the handwriting distribution program 2210 receives the content signal2240 indicative of a hand-formed mark on a surface and the label signal2242 indicative of a machine-distinguishable tag associated with thehand-formed mark. In response to the machine-distinguishable tag, thehandwriting distribution program 2210 distributes a representation ofthe handwriting to a data recipient. For example, the content signal2240 indicative of a hand-formed mark on a surface may include a contentsignal indicative of the hand-formed mark “Buy welcome home card” 1916on the surface 1912 of the document 1910 of FIG. 27. The label signal2242 indicative of a machine-distinguishable tag associated with thehand-formed mark may include a label signal indicative of themachine-distinguishable tag “●” 1924 a associated with the hand-formedmark. In response to the tag distribution table 2220, the handwritingdistribution program distributes the hand-formed mark “Buy welcome homecard” to the “Urgent tasks list” 2232.

FIG. 49 illustrates a partial view of an exemplary apparatus 2300. Theapparatus includes a means 2310 for receiving in a computing device asignal indicative of a handwriting on a surface. The apparatus includesa means 2320 for receiving in the computing device a signal indicativeof a machine-distinguishable tag associated by a user with thehandwriting, the machine-distinguishable tag being selected from atleast two machine-distinguishable tags. The apparatus also includes ameans 2330 for distributing a representation of the handwriting to adata recipient associated with the computing device in response to themachine-distinguishable tag.

FIG. 50 illustrates an exemplary operational flow 2350. After a startoperation, the operational flow moves to a mark description operation2360. The mark description operation 2360 generates a content signalindicative of a hand-formed mark on a surface. A description operation2370 generates a label signal indicative of a machine-distinguishabletag user associated with the hand-formed mark. Themachine-distinguishable tag is selected from at least twomachine-distinguishable tags. The operational flow then moves to an endoperation.

FIG. 51 illustrates a partial view of an environment 2400 in whichembodiments may be implemented. The environment includes a computingdevice 2405. In an embodiment, the computing device may include the thincomputing device 20 described in conjunction with FIG. 1, a handheldcomputing device, a portable computing device, a desktop computingdevice, and/or the general-purpose computing device 110 described inconjunction with FIG. 2. The computing device includes a computerstorage media. In an embodiment, the computer storage media may includea system memory, such as the system memory 130, and/or a non-removablenon-volatile computer storage product, such as the disk drive 141, bothof which are described in conjunction with FIG. 2. The computing devicemay include a data recipient file saved in the computer storage media.In an embodiment, the saved data recipient file may include the itemlist files 2230, book annotation files 2420, and/or another datarecipient 2412. In an embodiment illustrated in FIG. 51, the item listfiles include an operability to receive data. The item list files may bea member of a data recipient file class associated or associatable withany feature of the computing device, such as an operating system, anapplication, and/or a program. For example, the item list files may beassociated with Microsoft's OUTLOOK™ program.

In an embodiment, at least one file of the item list files includes atleast one representation of a hand-formed mark received by the file inresponse to a machine-distinguishable tag associated with a hand-formedmark, the machine-distinguishable tag selected from at least twomachine-distinguishable tags. For example, the at least onerepresentation of a hand-formed mark may include a representation of thehand-formed mark “Buy welcome home card” 1916 of the document 1910 ofFIG. 27. The machine-distinguishable tag associated with the hand-formedmark may include the machine-distinguishable tag “●” 1924 also of FIG.27. The hand-formed mark “Buy welcome home card” having been distributedto and received by the “Urgent tasks list” 2232 in response to themachine-distinguishable tag “●”.

The computing device may also include at least one program saved in thecomputer storage media. In an embodiment, the at least one programincludes a list manager program 2410, and/or another program 2414.

The list manager program 2410 includes instructions operable to performa process in the computing device 2405. The process includes receiving arequest for a content of a file, such as the item list file 2230, andsearching a computer storage medium, such as the computer storage media(130, 141), for the requested content of the file. The process alsoincludes outputting the at least one representation of a hand-formedmark. In an embodiment, the outputting the at least one representationof a hand-formed mark may include outputting the at least onerepresentation of a hand-formed mark to at least one of a display,and/or another program.

The environment 2400 also may include a display, illustrated as themonitor 191 described in conjunction with FIG. 2. The environment mayalso include a user input device 2440, which may include any user inputdevice. For example, the user input device may include any or all of theuser input devices described in conjunction with FIG. 2, such as themouse 161, the keyboard 162, the microphone 163, and/or atouch-sensitive device (not shown) associated with the monitor 191.

In operation of an embodiment, a user inputs a request for a content ofa computer-readable file using the user input device 2440. In anembodiment, the request for a content of a computer-readable file mayinclude a general request for a content of a high-level file, such as“show me a content related to the Iliad saved on the computer,” i.e.,the storage medium disk drive 141 associated with the computing device2405. In another embodiment, the request for a content of acomputer-readable file may include a low-level request, such as “show mea content of my urgent tasks list 2232.” The list manager program 2410receives the user originated request for content of a computer-readablefile. The list manager program 2410 searches the computer-readable fileor files for the requested content. The computer-readable file or filesincludes at least one representation of a hand-formed mark distributedto the computer-readable file in response to a machine-distinguishabletag associated with the hand-formed mark, the machine-distinguishabletag selected from at least two machine-distinguishable tags. Forexample, if the user originated request pertained to the urgent taskslist 2232, the list manager program may cause the urgent task list to besearched for the requested content. The list manager program outputs acontent corresponding to the request, the outputted content including atleast one representation of a hand formed mark. Continuing with theabove example of a request for content of the urgent task list, the listmanager program would output at least the representation of thehand-formed mark 1916 “Buy welcome home card.” In an embodiment, theoutput may be provided to the monitor 191 for display to the user. Inanother embodiment, the output may be exposed to the another program2414, and/or the another data recipient 2412.

FIG. 52 illustrates an exemplary operation flow 2500. After a startoperation, the operational flow moves to an accepting operation 2510.The accepting operation receives a request for a content of acomputer-readable file. A locate operation 2530 searches thecomputer-readable file for the requested content. The computer-readablefile including at least one representation of a hand-formed markdistributed to the computer-readable file in response to amachine-distinguishable tag associated with the hand-formed mark, themachine-distinguishable tag selected from at least twomachine-distinguishable tags. A presenting operation 2550 outputs the atleast one representation of a hand-formed mark. The operational flowthen moves to an end operation.

FIG. 53 illustrates an alternative embodiment of the exemplary operationflow 2500 of FIG. 52. The accepting operation 2510 may include at leastone additional operation. An additional operation may include anoperation 2512, an operation 2518, an operation 2522, and/or anoperation 2524. The operation 2512 receives a request for a content of acomputer-readable file having a relationship with a data recipient. Theoperation 2512 may include at least one additional operation. Anadditional operation may include an operation 2514, and/or an operation2516. The operation 2514 receives a request for a content of acomputer-readable file having a relationship with an item list file. Theoperation 2516 receives a request for a content of a computer-readablefile having a relationship with a program. The operation 2518 receives arequest from a user for a content of a computer-readable file. Theoperation 2522 receives a request for a content of a computer-readablefile saved in a computer storage media associated with a computingdevice. The operation 2524 receives a request for a representation of ahand-formed mark distributed to the computer-readable file in responseto a machine-distinguishable tag associated with the hand-formed mark.

FIG. 54 illustrates another alternative embodiment of the exemplaryoperation flow 2500 of FIG. 52. The locating operation 2530 may includeat least one additional operation. An additional operation may includean operation 2532, an operation 2534, an operation 2536, an operation253 8, and/or an operation 2542. The operation 2532 searches thecomputer-readable file for the requested content, the computer-readablefile including at least one representation of a handwritten mark. Theoperation 2534 searches the computer-readable file for the requestedcontent, the computer-readable file including at least onerepresentation of a handwriting. The operation 2536 searches thecomputer-readable file for the requested content, the computer-readablefile including at least one representation of a hand-formed character.At the operation 2538, the machine-distinguishable tag associated withthe hand-formed mark includes a hand-formed machine-distinguishable tagassociated with the hand-formed mark. At the operation 2542, themachine-distinguishable tag associated with the hand-formed markincludes a preformed machine-distinguishable tag associated with thehand-formed mark.

FIG. 55 illustrates a further alternative embodiment of the exemplaryoperation flow 2500 of FIG. 52. The presenting operation 2550 mayinclude at least one additional operation. An additional operation mayinclude an operation 2552, and/or 2554. The operation 2552 provides theat least one representation of a hand-formed mark to a user. Theoperation 2554 exposes the at least one representation of a hand-formedmark to a program.

FIG. 56 illustrates a partial view of an exemplary computer program 2560product. The computer program product includes a computer program 2580and a computer-readable signal-bearing medium 2570 bearing the computerprogram. The computer program product encodes the computer program ascomputer executable instructions for executing a computer process on acomputing device. The computer process receives a request forinformation saved in a computer-readable file. The computer-readablefile including at least one representation of a hand-formed markdistributed to the computer-readable file in response to amachine-distinguishable tag associated with the hand-formed mark Themachine-distinguishable tag being selected from at least twomachine-distinguishable tags. The computer process searches thecomputer-readable file for the requested information, and outputs the atleast one representation of a hand-formed mark

In an embodiment, the computer-readable signal-bearing medium 2570 mayinclude a computer storage medium 2572, which may be carried by acomputer-readable carrier (not shown). The computer-readablesignal-bearing medium may include a communications medium 2574. In afurther alternative embodiment, the computer-program 2570 may beimplemented in hardware, software, and/or firmware.

FIG. 57 illustrates a partial view of an exemplary device 2600. Theexemplary device includes a means 2610, a means 2620, and a means 2630.The means 2610 includes a means for receiving a request for a content ofa computer-readable file. The means 2620 includes a means for searchingthe computer-readable file for the requested content. Thecomputer-readable file including at least one representation of ahand-formed mark distributed to the computer-readable file in responseto a machine-distinguishable tag associated with the hand-formed mark,and the machine-distinguishable tag being selected from at least twomachine-distinguishable tags. The means 2630 includes a means forproviding the at least one representation of a hand-formed mark.

Those having skill in the art will recognize that the state of the arthas progressed to the point where there is little distinction leftbetween hardware and software implementations of aspects of systems; theuse of hardware or software is generally (but not always, in that incertain contexts the choice between hardware and software can becomesignificant) a design choice representing cost vs. efficiency tradeoffs.Those having skill in the art will appreciate that there are variousvehicles by which processes and/or systems and/or other technologiesdescribed herein can be effected (e.g., hardware, software, and/orfirmware), and that the preferred vehicle will vary with the context inwhich the processes and/or systems and/or other technologies aredeployed. For example, if an implementer determines that speed andaccuracy are paramount, the implementer may opt for a mainly hardwareand/or firmware vehicle; alternatively, if flexibility is paramount, theimplementer may opt for a mainly software implementation; or, yet againalternatively, the implementer may opt for some combination of hardware,software, and/or firmware. Hence, there are several possible vehicles bywhich the processes and/or devices and/or other technologies describedherein may be effected, none of which is inherently superior to theother in that any vehicle to be utilized is a choice dependent upon thecontext in which the vehicle will be deployed and the specific concerns(e.g., speed, flexibility, or predictability) of the implementer, any ofwhich may vary. Those skilled in the art will recognize that opticalaspects of implementations will require optically-oriented hardware,software, and or firmware.

The foregoing detailed description has set forth various embodiments ofthe devices and/or processes via the use of block diagrams, flowdiagrams, operation diagrams, flowcharts, illustrations, and/orexamples. Insofar as such block diagrams, operation diagrams,flowcharts, illustrations, and/or examples contain one or more functionsand/or operations, it will be understood by those within the art thateach function and/or operation within such block diagrams, operationdiagrams, flowcharts, illustrations, or examples can be implemented,individually and/or collectively, by a wide range of hardware, software,firmware, or virtually any combination thereof. In one embodiment,several portions of the subject matter described herein may beimplemented via Application Specific Integrated Circuits (ASICs), FieldProgrammable Gate Arrays (FPGAs), digital signal processors (DSPs), orother integrated formats. However, those skilled in the art willrecognize that some aspects of the embodiments disclosed herein, inwhole or in part, can be equivalently implemented in standard integratedcircuits, as one or more computer programs running on one or morecomputers (e.g., as one or more programs running on one or more computersystems), as one or more programs running on one or more processors(e.g., as one or more programs running on one or more microprocessors),as firmware, or as virtually any combination thereof, and that designingthe circuitry and/or writing the code for the software and or firmwarewould be well within the skill of one of skill in the art in light ofthis disclosure. In addition, those skilled in the art will appreciatethat the mechanisms of the subject matter described herein are capableof being distributed as a program product in a variety of forms, andthat an illustrative embodiment of the subject matter described hereinapplies equally regardless of the particular type of signal bearingmedia used to actually carry out the distribution. Examples of a signalbearing media include, but are not limited to, the following: recordabletype media such as floppy disks, hard disk drives, CD ROMs, digitaltape, and computer memory; and transmission type media such as digitaland analog communication links using TDM or IP based communication links(e.g., packet links).

It will be understood by those within the art that, in general, termsused herein, and especially in the appended claims (e.g., bodies of theappended claims) are generally intended as “open” terms (e.g., the term“including” should be interpreted as “including but not limited to,” theterm “having” should be interpreted as “having at least,” the term“includes” should be interpreted as “includes but is not limited to,”etc.). It will be further understood by those within the art that if aspecific number of an introduced claim recitation is intended, such anintent will be explicitly recited in the claim, and in the absence ofsuch recitation no such intent is present. For example, as an aid tounderstanding, the following appended claims may contain usage of theintroductory phrases “at least one” and “one or more” to introduce claimrecitations. However, the use of such phrases should not be construed toimply that the introduction of a claim recitation by the indefinitearticles “a” or “an” limits any particular claim containing suchintroduced claim recitation to inventions containing only one suchrecitation, even when the same claim includes the introductory phrases“one or more” or “at least one” and indefinite articles such as “a” or“an” (e.g., “a” and/or “an” should typically be interpreted to mean “atleast one” or “one or more”); the same holds true for the use ofdefinite articles used to introduce claim recitations. In addition, evenif a specific number of an introduced claim recitation is explicitlyrecited, those skilled in the art will recognize that such recitationshould typically be interpreted to mean at least the recited number(e.g., the bare recitation of “two recitations,” without othermodifiers, typically means at least two recitations, or two or morerecitations). Furthermore, in those instances where a conventionanalogous to “at least one of A, B, and C, etc.” is used, in generalsuch a construction is intended in the sense one having skill in the artwould understand the convention (e.g., “a system having at least one ofA, B, and C” would include but not be limited to systems that have Aalone, B alone, C alone, A and B together, A and C together, B and Ctogether, and/or A, B, and C together, etc.). In those instances where aconvention analogous to “at least one of A, B, or C, etc.” is used, ingeneral such a construction is intended in the sense one having skill inthe art would understand the convention (e.g., “a system having at leastone of A, B, or C” would include but not be limited to systems that haveA alone, B alone, C alone, A and B together, A and C together, B and Ctogether, and/or A, B, and C together, etc.).

The herein described aspects depict different components containedwithin, or connected with, different other components. It is to beunderstood that such depicted architectures are merely exemplary, andthat in fact many other architectures can be implemented which achievethe same functionality. In a conceptual sense, any arrangement ofcomponents to achieve the same functionality is effectively “associated”such that the desired functionality is achieved. Hence, any twocomponents herein combined to achieve a particular functionality can beseen as “associated with” each other such that the desired functionalityis achieved, irrespective of architectures or intermedial components.Likewise, any two components so associated can also be viewed as being“operably connected,” or “operably coupled,” to each other to achievethe desired functionality. Any two components capable of being soassociated can also be viewed as being “operably couplable” to eachother to achieve the desired functionality. Specific examples ofoperably couplable include but are not limited to physically mateableand/or physically interacting components and/or wirelessly interactableand/or wirelessly interacting components.

While particular aspects of the present subject matter described hereinhave been shown and described, it will be apparent to those skilled inthe art that, based upon the teachings herein, changes and modificationsmay be made without departing from this subject matter described hereinand its broader aspects and, therefore, the appended claims are toencompass within their scope all such changes and modifications as arewithin the true spirit and scope of this subject matter describedherein. Furthermore, it is to be understood that the invention is solelydefined by the appended claims.

1. A method comprising: receiving a request for a content of acomputer-readable file; searching the computer-readable file for therequested content, the computer-readable file including at least onerepresentation of a hand-formed mark distributed to thecomputer-readable file in response to a machine-distinguishable tagassociated with the hand-formed mark, the machine-distinguishable tagselected from at least two machine-distinguishable tags; and outputtingthe at least one representation of a hand-formed mark.
 2. The method ofclaim 1, wherein the receiving a request for a content of acomputer-readable file includes receiving a request for a content of acomputer-readable file having a relationship with a data recipient. 3.The method of claim 2, wherein the receiving a request for a content ofa computer-readable file having a relationship with a data recipientincludes receiving a request for a content of a computer-readable filehaving a relationship with an item list file.
 4. The method of claim 2,wherein the receiving a request for a content of a computer-readablefile having a relationship with a data recipient includes receiving arequest for a content of a computer-readable file having a relationshipwith a program.
 5. The method of claim 1, wherein the receiving arequest for a content of a computer-readable file includes receiving arequest from a user for a content of a computer-readable file.
 6. Themethod of claim 1, wherein the receiving a request for a content of acomputer-readable file includes receiving a request for a content of acomputer-readable file saved in a computer storage media associated witha computing device.
 7. The method of claim 1, wherein the receiving arequest for a content of a computer-readable file includes receiving arequest for a representation of a hand-formed mark distributed to thecomputer-readable file in response to a machine-distinguishable tagassociated with the hand-formed mark.
 8. The method of claim 1, whereinthe searching the computer-readable file for the requested content, thecomputer-readable file including at least one representation of ahand-formed mark includes searching the computer-readable file for therequested content, the computer-readable file including at least onerepresentation of a handwritten mark.
 9. The method of claim 1, whereinthe searching the computer-readable file for the requested content, thecomputer-readable file including at least one representation of ahand-formed mark includes searching the computer-readable file for therequested content, the computer-readable file including at least onerepresentation of a handwriting.
 10. The method of claim 1, wherein thesearching the computer-readable file for the requested content, thecomputer-readable file including at least one representation of ahand-formed mark includes searching the computer-readable file for therequested content, the computer-readable file including at least onerepresentation of a hand-formed character.
 11. The method of claim 1,wherein the machine-distinguishable tag associated with the hand-formedmark includes a hand-formed machine-distinguishable tag associated withthe hand-formed mark.
 12. The method of claim 1, wherein themachine-distinguishable tag associated with the hand-formed markincludes a preformed machine-distinguishable tag associated with thehand-formed mark.
 13. The method of claim 1, wherein the outputting theat least one representation of a hand-formed mark includes providing theat least one representation of a hand-formed mark to a user.
 14. Themethod of claim 1, wherein the outputting the at least onerepresentation of a hand-formed mark includes exposing the at least onerepresentation of a hand-formed mark to a program.
 15. A computerprogram product, comprising: (a) program instructions operable toperform a process in a computer system, the process comprising:receiving a request for information saved in a computer-readable file,the computer-readable file including at least one representation of ahand-formed mark distributed to the computer-readable file in responseto a machine-distinguishable tag associated with the hand-formed mark,the machine-distinguishable tag selected from at least twomachine-distinguishable tags; searching the computer-readable file forthe requested information; and outputting the at least onerepresentation of a hand-formed mark; and (b) a computer-readablesignal-bearing medium bearing the program instructions.
 16. The computerprogram product of claim 15, wherein the computer-readablesignal-bearing medium includes a computer storage medium.
 17. Thecomputer program product of claim 15, wherein the computer-readablesignal-bearing medium includes a communication medium.
 18. A apparatuscomprising: a computing device having a computer storage medium and alist manager program; a data recipient file saved on the computerstorage medium and including at least one representation of ahand-formed mark received by the data recipient file in response to amachine-distinguishable tag associated with a hand-formed mark, themachine-distinguishable tag selected from at least twomachine-distinguishable tags; and the list manager program that includesinstructions operable to perform a process in the computing device, theprocess comprising: receiving a request for a content of the datarecipient file; searching the computer storage medium for the requestedcontent of the data recipient file; and outputting the at least onerepresentation of a hand-formed mark.
 19. The apparatus of claim 18,wherein the computing device includes at least one of a thin computingdevice, a handheld computing device, a portable computing device, and/ora desktop computing device.
 20. The apparatus of claim 18, wherein theoutputting the at least one representation of a hand-formed markincludes outputting the at least one representation of a hand-formedmark to at least one of a display, and/or another program.
 21. A devicecomprising: means for receiving a request for a content of acomputer-readable file; means for searching the computer-readable filefor the requested content, the computer-readable file including at leastone representation of a hand-formed mark distributed to thecomputer-readable file in response to a machine-distinguishable tagassociated with the hand-formed mark, the machine-distinguishable tagselected from at least two machine-distinguishable tags; and means forproviding the at least one representation of a hand-formed mark.